Max level shown:
1 [ Mar 21 18:03:39 Enabled. ]
2 [ Mar 21 18:03:39 Rereading configuration. ]
3 [ Mar 21 18:03:41 Rereading configuration. ]
4 [ Mar 21 18:03:52 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/omicron-nexus/bin/nexus /var/svc/manifest/site/nexus/config.toml &"). ]
5 [ Mar 21 18:03:52 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72024-03-21T18:03:53.166ZDEBGnexus: registered DTrace probes
82024-03-21T18:03:53.168ZINFOnexus: setting up nexus server file = nexus/src/lib.rs:90
92024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Action file = nexus/db-queries/src/authz/oso_generic.rs:68
102024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = AnyActor file = nexus/db-queries/src/authz/oso_generic.rs:68
112024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = AuthenticatedActor file = nexus/db-queries/src/authz/oso_generic.rs:68
122024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = BlueprintConfig file = nexus/db-queries/src/authz/oso_generic.rs:68
132024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Database file = nexus/db-queries/src/authz/oso_generic.rs:68
142024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = DnsConfig file = nexus/db-queries/src/authz/oso_generic.rs:68
152024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Fleet file = nexus/db-queries/src/authz/oso_generic.rs:68
162024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Inventory file = nexus/db-queries/src/authz/oso_generic.rs:68
172024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = IpPoolList file = nexus/db-queries/src/authz/oso_generic.rs:68
182024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = ConsoleSessionList file = nexus/db-queries/src/authz/oso_generic.rs:68
192024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequestList file = nexus/db-queries/src/authz/oso_generic.rs:68
202024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = SiloCertificateList file = nexus/db-queries/src/authz/oso_generic.rs:68
212024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = SiloIdentityProviderList file = nexus/db-queries/src/authz/oso_generic.rs:68
222024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = SiloUserList file = nexus/db-queries/src/authz/oso_generic.rs:68
232024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Project file = nexus/db-queries/src/authz/oso_generic.rs:68
242024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Disk file = nexus/db-queries/src/authz/oso_generic.rs:68
252024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Snapshot file = nexus/db-queries/src/authz/oso_generic.rs:68
262024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = ProjectImage file = nexus/db-queries/src/authz/oso_generic.rs:68
272024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Instance file = nexus/db-queries/src/authz/oso_generic.rs:68
282024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = IpPool file = nexus/db-queries/src/authz/oso_generic.rs:68
292024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = InstanceNetworkInterface file = nexus/db-queries/src/authz/oso_generic.rs:68
302024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Vpc file = nexus/db-queries/src/authz/oso_generic.rs:68
312024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = VpcRouter file = nexus/db-queries/src/authz/oso_generic.rs:68
322024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = RouterRoute file = nexus/db-queries/src/authz/oso_generic.rs:68
332024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = VpcSubnet file = nexus/db-queries/src/authz/oso_generic.rs:68
342024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = FloatingIp file = nexus/db-queries/src/authz/oso_generic.rs:68
352024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Image file = nexus/db-queries/src/authz/oso_generic.rs:68
362024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = SiloImage file = nexus/db-queries/src/authz/oso_generic.rs:68
372024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = AddressLot file = nexus/db-queries/src/authz/oso_generic.rs:68
382024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Blueprint file = nexus/db-queries/src/authz/oso_generic.rs:68
392024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = LoopbackAddress file = nexus/db-queries/src/authz/oso_generic.rs:68
402024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Certificate file = nexus/db-queries/src/authz/oso_generic.rs:68
412024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = ConsoleSession file = nexus/db-queries/src/authz/oso_generic.rs:68
422024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequest file = nexus/db-queries/src/authz/oso_generic.rs:68
432024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = DeviceAccessToken file = nexus/db-queries/src/authz/oso_generic.rs:68
442024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = PhysicalDisk file = nexus/db-queries/src/authz/oso_generic.rs:68
452024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Rack file = nexus/db-queries/src/authz/oso_generic.rs:68
462024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = RoleBuiltin file = nexus/db-queries/src/authz/oso_generic.rs:68
472024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = SshKey file = nexus/db-queries/src/authz/oso_generic.rs:68
482024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Silo file = nexus/db-queries/src/authz/oso_generic.rs:68
492024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = SiloUser file = nexus/db-queries/src/authz/oso_generic.rs:68
502024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = SiloGroup file = nexus/db-queries/src/authz/oso_generic.rs:68
512024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = IdentityProvider file = nexus/db-queries/src/authz/oso_generic.rs:68
522024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = SamlIdentityProvider file = nexus/db-queries/src/authz/oso_generic.rs:68
532024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Sled file = nexus/db-queries/src/authz/oso_generic.rs:68
542024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = TufRepo file = nexus/db-queries/src/authz/oso_generic.rs:68
552024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = TufArtifact file = nexus/db-queries/src/authz/oso_generic.rs:68
562024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Zpool file = nexus/db-queries/src/authz/oso_generic.rs:68
572024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = Service file = nexus/db-queries/src/authz/oso_generic.rs:68
582024-03-21T18:03:53.181ZINFOnexus (ServerContext): registering Oso class class = UserBuiltin file = nexus/db-queries/src/authz/oso_generic.rs:68
592024-03-21T18:03:53.181ZINFOnexus (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n file = nexus/db-queries/src/authz/oso_generic.rs:89
602024-03-21T18:03:53.197ZINFOnexus (ServerContext): Setting up resolver using DNS servers for subnet: Ipv6Subnet { net: Ipv6Net(Ipv6Network { addr: fd00:1122:3344::, prefix: 48 }) } file = nexus/src/context.rs:161
612024-03-21T18:03:53.200ZINFOnexus (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
622024-03-21T18:03:53.202ZINFOnexus (ServerContext): Accessing DB url from DNS file = nexus/src/context.rs:190
632024-03-21T18:03:53.217ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("0ec8aadf-5087-4bca-8642-8e50b6d5e739.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2fd71886-b669-4748-b128-312cefeef46f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5212b16c-c411-4b79-897f-c1d2ce097b13.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("962c0adc-25a8-4b52-9c01-db6860e37774.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b5b11a81-6066-4bf8-b39d-d567e6f605f9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b5b11a81-6066-4bf8-b39d-d567e6f605f9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 938598403 } })
642024-03-21T18:03:53.220ZINFOnexus (ServerContext): DB addresses: [fd00:1122:3344:101::7]:32221,[fd00:1122:3344:101::5]:32221,[fd00:1122:3344:101::4]:32221,[fd00:1122:3344:101::3]:32221,[fd00:1122:3344:101::6]:32221 file = nexus/src/context.rs:220
652024-03-21T18:03:53.221ZINFOnexus (db::Pool): database connection pool database_url = postgresql://root@[fd00:1122:3344:101::7]:32221,[fd00:1122:3344:101::5]:32221,[fd00:1122:3344:101::4]:32221,[fd00:1122:3344:101::3]:32221,[fd00:1122:3344:101::6]:32221/omicron?sslmode=disable file = nexus/db-queries/src/db/pool.rs:69
662024-03-21T18:03:53.286ZINFOnexus: Database schema version is up to date desired_version = 46.0.0 file = nexus/db-queries/src/db/datastore/db_metadata.rs:63 found_version = 46.0.0
672024-03-21T18:03:53.289ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 17320584 } })
682024-03-21T18:03:53.289ZINFOnexus (SEC): SEC running file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.0/src/sec.rs:811 sec_id = 63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
692024-03-21T18:03:53.290ZINFOnexus (Nexus): Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:974
702024-03-21T18:03:53.341ZINFOnexus (Nexus): determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
712024-03-21T18:03:53.341ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
722024-03-21T18:03:53.344ZDEBGnexus (MgsClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2b42dc7c-36c8-4e86-97bf-414fe5141531", "content-length": "26", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
732024-03-21T18:03:53.344ZINFOnexus (Nexus): identified switch slot for dendrite zone file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
742024-03-21T18:03:53.344ZINFOnexus (Nexus): completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
752024-03-21T18:03:53.375ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 103384766 } })
762024-03-21T18:03:53.376ZINFOnexus (Nexus): Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:974
772024-03-21T18:03:53.407ZINFOnexus (Nexus): determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
782024-03-21T18:03:53.407ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
792024-03-21T18:03:53.407ZDEBGnexus (MgsClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f53b713d-2ee3-4b63-ac6a-c6399be4881e", "content-length": "26", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
802024-03-21T18:03:53.407ZINFOnexus (Nexus): identified switch slot for dendrite zone file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
812024-03-21T18:03:53.408ZINFOnexus (Nexus): completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
822024-03-21T18:03:53.444ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
832024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
842024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
852024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
862024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
872024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
882024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
892024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
902024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
912024-03-21T18:03:53.451ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:61 reason = no config nor servers
922024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
932024-03-21T18:03:53.451ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:61 reason = no config nor servers
942024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
952024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
962024-03-21T18:03:53.451ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
972024-03-21T18:03:53.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
982024-03-21T18:03:53.452ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
992024-03-21T18:03:53.453ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1002024-03-21T18:03:53.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1012024-03-21T18:03:53.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1022024-03-21T18:03:53.453ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
1032024-03-21T18:03:53.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1042024-03-21T18:03:53.455ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
1052024-03-21T18:03:53.455ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
1062024-03-21T18:03:53.455ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
1072024-03-21T18:03:53.455ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 727.344\u{b5}s iteration = 1
1082024-03-21T18:03:53.461ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 188668314 } })
1092024-03-21T18:03:53.463ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external dns_group = external file = nexus/src/app/background/dns_servers.rs:113
1102024-03-21T18:03:53.463ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 11.636429ms iteration = 1
1112024-03-21T18:03:53.468ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1122024-03-21T18:03:53.468ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 491.099\u{b5}s iteration = 1
1132024-03-21T18:03:53.468ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
1142024-03-21T18:03:53.468ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:69 reason = no config
1152024-03-21T18:03:53.468ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 4.8\u{b5}s iteration = 2
1162024-03-21T18:03:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1172024-03-21T18:03:53.472ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 465.358\u{b5}s iteration = 1
1182024-03-21T18:03:53.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1192024-03-21T18:03:53.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1202024-03-21T18:03:53.475ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1212024-03-21T18:03:53.475ZDEBGnexus (DataLoader): attempting to create built-in users
1222024-03-21T18:03:53.475ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1232024-03-21T18:03:53.475ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1242024-03-21T18:03:53.476ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
1252024-03-21T18:03:53.476ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
1262024-03-21T18:03:53.476ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
1272024-03-21T18:03:53.476ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
1282024-03-21T18:03:53.476ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:54 reason = no blueprint
1292024-03-21T18:03:53.476ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1302024-03-21T18:03:53.476ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1312024-03-21T18:03:53.476ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 31.61\u{b5}s iteration = 1
1322024-03-21T18:03:53.476ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
1332024-03-21T18:03:53.476ZINFOnexus (SagaRecoverer): start saga recovery file = nexus/db-queries/src/db/saga_recovery.rs:77
1342024-03-21T18:03:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1352024-03-21T18:03:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1362024-03-21T18:03:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1372024-03-21T18:03:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1382024-03-21T18:03:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
1392024-03-21T18:03:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1402024-03-21T18:03:53.478ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 206204568 } })
1412024-03-21T18:03:53.482ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal dns_group = internal file = nexus/src/app/background/dns_servers.rs:113
1422024-03-21T18:03:53.482ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 30.824642ms iteration = 1
1432024-03-21T18:03:53.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
1442024-03-21T18:03:53.482ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:69 reason = no config
1452024-03-21T18:03:53.482ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 3.56\u{b5}s iteration = 2
1462024-03-21T18:03:53.487ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1472024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /artifacts/{kind}/{name}/{version}
1482024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /bgtasks
1492024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /bgtasks/{bgtask_name}
1502024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request
1512024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped
1522024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress
1532024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish
1542024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start
1552024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /deployment/blueprints/all
1562024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = DELETE path = /deployment/blueprints/all/{blueprint_id}
1572024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /deployment/blueprints/all/{blueprint_id}
1582024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /deployment/blueprints/generate-from-collection
1592024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /deployment/blueprints/regenerate
1602024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /deployment/blueprints/target
1612024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /deployment/blueprints/target
1622024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /deployment/blueprints/target/enabled
1632024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent
1642024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /disks/{disk_id}
1652024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /instances/{instance_id}
1662024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /metrics/collect/{producer_id}
1672024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /metrics/collectors
1682024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /metrics/producers
1692024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /nat/ipv4/changeset/{from_gen}
1702024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = DELETE path = /physical-disk
1712024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /physical-disk
1722024-03-21T18:03:53.492ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /probes/{sled}
1732024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /racks/{rack_id}/initialization-complete
1742024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /sagas
1752024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /sagas/{saga_id}
1762024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /sled-agents/{sled_id}
1772024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /sled-agents/{sled_id}
1782024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /sled-agents/{sled_id}/firewall-rules-update
1792024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /sled-agents/{sled_id}/zpools/{zpool_id}
1802024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /sleds/add
1812024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /sleds/expunge
1822024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = GET path = /sleds/uninitialized
1832024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = PUT path = /switch/{switch_id}
1842024-03-21T18:03:53.493ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::b]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent
1852024-03-21T18:03:53.493ZINFOnexus (dropshot_internal): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:195 local_addr = [fd00:1122:3344:101::b]:12221
1862024-03-21T18:03:53.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Err(Forbidden)
1872024-03-21T18:03:53.493ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Forbidden file = nexus/src/app/background/external_endpoints.rs:55
1882024-03-21T18:03:53.493ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 41.691937ms iteration = 1
1892024-03-21T18:03:53.495ZDEBGnexus (dropshot_internal): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::b]:12221
1902024-03-21T18:03:53.496ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1912024-03-21T18:03:53.496ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1922024-03-21T18:03:53.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1932024-03-21T18:03:53.510ZINFOnexus (SagaRecoverer): listed sagas (0 total) file = nexus/db-queries/src/db/saga_recovery.rs:113
1942024-03-21T18:03:53.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1952024-03-21T18:03:53.513ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 241568852 } })
1962024-03-21T18:03:53.515ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
1972024-03-21T18:03:53.517ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 245445421 } })
1982024-03-21T18:03:53.519ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
1992024-03-21T18:03:53.522ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
2002024-03-21T18:03:53.522ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 71.060604ms iteration = 1
2012024-03-21T18:03:53.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2022024-03-21T18:03:53.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2032024-03-21T18:03:53.524ZWARNnexus: Cannot look up rack: Object (of type ById(11ad3243-b088-4dcc-87e3-5a6072b1f137)) not found: rack file = nexus/src/app/rack.rs:653
2042024-03-21T18:03:53.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2052024-03-21T18:03:53.525ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2062024-03-21T18:03:53.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Err(Forbidden)
2072024-03-21T18:03:53.527ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Forbidden file = nexus/src/app/background/dns_config.rs:72
2082024-03-21T18:03:53.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 75.8812ms iteration = 1
2092024-03-21T18:03:53.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Err(Forbidden)
2102024-03-21T18:03:53.527ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Forbidden file = nexus/src/app/background/blueprint_load.rs:66
2112024-03-21T18:03:53.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 76.297148ms iteration = 1
2122024-03-21T18:03:53.528ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2132024-03-21T18:03:53.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Err(Forbidden)
2142024-03-21T18:03:53.528ZERROnexus (BackgroundTasks): failed to retrieve racks from database background_task = switch_port_config_manager error = Forbidden file = nexus/src/app/background/sync_switch_configuration.rs:234
2152024-03-21T18:03:53.528ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 51.983042ms iteration = 1
2162024-03-21T18:03:53.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2172024-03-21T18:03:53.528ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2182024-03-21T18:03:53.528ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Err(Forbidden)
2192024-03-21T18:03:53.528ZWARNnexus (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: pruning old collections: Forbidden file = nexus/src/app/background/inventory_collection.rs:72
2202024-03-21T18:03:53.528ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 52.444251ms iteration = 1
2212024-03-21T18:03:53.529ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2222024-03-21T18:03:53.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Err(Forbidden)
2232024-03-21T18:03:53.532ZERROnexus (BackgroundTasks): failed to collect inventory background_task = service_zone_nat_tracker error = Forbidden file = nexus/src/app/background/sync_service_zone_nat.rs:71
2242024-03-21T18:03:53.532ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 56.071625ms iteration = 1
2252024-03-21T18:03:53.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Err(Forbidden)
2262024-03-21T18:03:53.532ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Forbidden file = nexus/src/app/background/dns_config.rs:72
2272024-03-21T18:03:53.532ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 81.090673ms iteration = 1
2282024-03-21T18:03:53.532ZINFOnexus (DataLoader): created 5 built-in users file = nexus/db-queries/src/db/datastore/silo_user.rs:394
2292024-03-21T18:03:53.532ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2302024-03-21T18:03:53.533ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2312024-03-21T18:03:53.533ZDEBGnexus (DataLoader): attempting to create built-in roles
2322024-03-21T18:03:53.533ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2332024-03-21T18:03:53.533ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2342024-03-21T18:03:53.547ZINFOnexus (DataLoader): created 10 built-in roles file = nexus/db-queries/src/db/datastore/role.rs:87
2352024-03-21T18:03:53.547ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2362024-03-21T18:03:53.548ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2372024-03-21T18:03:53.548ZDEBGnexus (DataLoader): attempting to create built-in role assignments
2382024-03-21T18:03:53.548ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2392024-03-21T18:03:53.548ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2402024-03-21T18:03:53.551ZINFOnexus (DataLoader): created 4 built-in role assignments file = nexus/db-queries/src/db/datastore/role.rs:116
2412024-03-21T18:03:53.551ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2422024-03-21T18:03:53.552ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2432024-03-21T18:03:53.552ZDEBGnexus (DataLoader): attempting to create built-in silos
2442024-03-21T18:03:53.552ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2452024-03-21T18:03:53.552ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2462024-03-21T18:03:53.605ZINFOnexus (DataLoader): created 0 built-in silos file = nexus/db-queries/src/db/datastore/silo.rs:95
2472024-03-21T18:03:53.605ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2482024-03-21T18:03:53.605ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2492024-03-21T18:03:53.607ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2502024-03-21T18:03:53.608ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2512024-03-21T18:03:53.608ZDEBGnexus (DataLoader): attempting to create built-in projects
2522024-03-21T18:03:53.608ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2532024-03-21T18:03:53.608ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2542024-03-21T18:03:53.610ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2552024-03-21T18:03:53.610ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2562024-03-21T18:03:53.611ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2572024-03-21T18:03:53.611ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2582024-03-21T18:03:53.612ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2592024-03-21T18:03:53.614ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2602024-03-21T18:03:53.614ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2612024-03-21T18:03:53.614ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2622024-03-21T18:03:53.615ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2632024-03-21T18:03:53.615ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2642024-03-21T18:03:53.615ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2652024-03-21T18:03:53.617ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2662024-03-21T18:03:53.617ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2672024-03-21T18:03:53.617ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2682024-03-21T18:03:53.640ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
2692024-03-21T18:03:53.640ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2702024-03-21T18:03:53.640ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4cc124fc-0c0e-416c-a1ce-08a59b31ab1a", "content-length": "26", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
2712024-03-21T18:03:53.640ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
2722024-03-21T18:03:53.640ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2732024-03-21T18:03:53.641ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "58708f77-d98b-4a52-8a63-7181bee931c0", "content-length": "26", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
2742024-03-21T18:03:53.643ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2752024-03-21T18:03:53.643ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2762024-03-21T18:03:53.644ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2772024-03-21T18:03:53.644ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2782024-03-21T18:03:53.645ZINFOnexus (DataLoader): created built-in services project file = nexus/db-queries/src/db/datastore/project.rs:122
2792024-03-21T18:03:53.645ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2802024-03-21T18:03:53.646ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2812024-03-21T18:03:53.646ZDEBGnexus (DataLoader): attempting to create built-in VPCs
2822024-03-21T18:03:53.646ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2832024-03-21T18:03:53.646ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2842024-03-21T18:03:53.659ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2852024-03-21T18:03:53.660ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2862024-03-21T18:03:53.679ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2872024-03-21T18:03:53.680ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2882024-03-21T18:03:53.680ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2892024-03-21T18:03:53.680ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2902024-03-21T18:03:53.681ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2912024-03-21T18:03:53.682ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2922024-03-21T18:03:53.683ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2932024-03-21T18:03:53.685ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2942024-03-21T18:03:53.685ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2952024-03-21T18:03:53.685ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2962024-03-21T18:03:53.686ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2972024-03-21T18:03:53.686ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2982024-03-21T18:03:53.687ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2992024-03-21T18:03:53.687ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3002024-03-21T18:03:53.688ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3012024-03-21T18:03:53.690ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
3022024-03-21T18:03:53.690ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3032024-03-21T18:03:53.690ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3042024-03-21T18:03:53.704ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3052024-03-21T18:03:53.704ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "23b26053-83b7-434d-bed3-991733409da8", "content-length": "1", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
3062024-03-21T18:03:53.705ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 254.377379ms iteration = 1
3072024-03-21T18:03:53.706ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3082024-03-21T18:03:53.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3092024-03-21T18:03:53.707ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 255.860696ms iteration = 1
3102024-03-21T18:03:53.708ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3112024-03-21T18:03:53.708ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3122024-03-21T18:03:53.717ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3132024-03-21T18:03:53.717ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3142024-03-21T18:03:53.717ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3152024-03-21T18:03:53.718ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3162024-03-21T18:03:53.718ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3172024-03-21T18:03:53.719ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3182024-03-21T18:03:53.719ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3192024-03-21T18:03:53.722ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3202024-03-21T18:03:53.722ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3212024-03-21T18:03:53.722ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3222024-03-21T18:03:53.732ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3232024-03-21T18:03:53.732ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3242024-03-21T18:03:53.733ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3252024-03-21T18:03:53.733ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3262024-03-21T18:03:53.734ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3272024-03-21T18:03:53.734ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3282024-03-21T18:03:53.735ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3292024-03-21T18:03:53.738ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3302024-03-21T18:03:53.738ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3312024-03-21T18:03:53.738ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3322024-03-21T18:03:53.758ZDEBGnexus (DataLoader): attempting to create built-in VPC firewall rules
3332024-03-21T18:03:53.758ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3342024-03-21T18:03:53.758ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3352024-03-21T18:03:53.759ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3362024-03-21T18:03:53.759ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372024-03-21T18:03:53.760ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3382024-03-21T18:03:53.760ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3392024-03-21T18:03:53.761ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3402024-03-21T18:03:53.761ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3412024-03-21T18:03:53.762ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3422024-03-21T18:03:53.762ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3432024-03-21T18:03:53.763ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3442024-03-21T18:03:53.763ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3452024-03-21T18:03:53.764ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3462024-03-21T18:03:53.766ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3472024-03-21T18:03:53.766ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3482024-03-21T18:03:53.766ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3492024-03-21T18:03:53.767ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3502024-03-21T18:03:53.767ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3512024-03-21T18:03:53.768ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3522024-03-21T18:03:53.768ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3532024-03-21T18:03:53.769ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3542024-03-21T18:03:53.771ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3552024-03-21T18:03:53.771ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3562024-03-21T18:03:53.772ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3572024-03-21T18:03:53.783ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3582024-03-21T18:03:53.783ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3592024-03-21T18:03:53.784ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3602024-03-21T18:03:53.784ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3612024-03-21T18:03:53.785ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3622024-03-21T18:03:53.785ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3632024-03-21T18:03:53.786ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3642024-03-21T18:03:53.788ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3652024-03-21T18:03:53.788ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3662024-03-21T18:03:53.788ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3672024-03-21T18:03:53.815ZINFOnexus (DataLoader): created built-in VPC firewall rules file = nexus/db-queries/src/db/datastore/vpc.rs:218
3682024-03-21T18:03:53.815ZDEBGnexus (DataLoader): attempting to create built-in VPC Subnets
3692024-03-21T18:03:53.815ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3702024-03-21T18:03:53.815ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3712024-03-21T18:03:53.826ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3722024-03-21T18:03:53.826ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3732024-03-21T18:03:53.827ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3742024-03-21T18:03:53.827ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3752024-03-21T18:03:53.828ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3762024-03-21T18:03:53.828ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3772024-03-21T18:03:53.829ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3782024-03-21T18:03:53.829ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3792024-03-21T18:03:53.830ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3802024-03-21T18:03:53.830ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3812024-03-21T18:03:53.831ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3822024-03-21T18:03:53.833ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3832024-03-21T18:03:53.833ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3842024-03-21T18:03:53.833ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3852024-03-21T18:03:53.849ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3862024-03-21T18:03:53.849ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3872024-03-21T18:03:53.849ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3882024-03-21T18:03:53.850ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3892024-03-21T18:03:53.850ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3902024-03-21T18:03:53.850ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3912024-03-21T18:03:53.851ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3922024-03-21T18:03:53.853ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3932024-03-21T18:03:53.859ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3942024-03-21T18:03:53.859ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952024-03-21T18:03:53.860ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3962024-03-21T18:03:53.860ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3972024-03-21T18:03:53.861ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3982024-03-21T18:03:53.861ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992024-03-21T18:03:53.862ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4002024-03-21T18:03:53.862ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4012024-03-21T18:03:53.862ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4022024-03-21T18:03:53.865ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4032024-03-21T18:03:53.869ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4042024-03-21T18:03:53.870ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4052024-03-21T18:03:53.870ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4062024-03-21T18:03:53.870ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4072024-03-21T18:03:53.871ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4082024-03-21T18:03:53.871ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4092024-03-21T18:03:53.872ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4102024-03-21T18:03:53.872ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4112024-03-21T18:03:53.873ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4122024-03-21T18:03:53.875ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4132024-03-21T18:03:53.881ZINFOnexus (DataLoader): created built-in services vpc subnets file = nexus/db-queries/src/db/datastore/vpc.rs:261
4142024-03-21T18:03:53.881ZINFOnexus (DataLoader): created built-in services vpc file = nexus/db-queries/src/db/datastore/vpc.rs:164
4152024-03-21T18:03:53.881ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4162024-03-21T18:03:53.882ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4172024-03-21T18:03:53.882ZDEBGnexus (DataLoader): attempting to create silo users
4182024-03-21T18:03:53.882ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4192024-03-21T18:03:53.882ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4202024-03-21T18:03:53.897ZINFOnexus (DataLoader): created 2 silo users file = nexus/db-queries/src/db/datastore/silo_user.rs:419
4212024-03-21T18:03:53.897ZDEBGnexus (DataLoader): attempting to create silo user role assignments
4222024-03-21T18:03:53.897ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4232024-03-21T18:03:53.898ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4242024-03-21T18:03:53.903ZINFOnexus (DataLoader): created 2 silo user role assignments file = nexus/db-queries/src/db/datastore/silo_user.rs:444
4252024-03-21T18:03:53.903ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4262024-03-21T18:03:53.903ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4272024-03-21T18:03:53.906ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4282024-03-21T18:03:53.906ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4292024-03-21T18:03:53.918ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4302024-03-21T18:03:53.918ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4312024-03-21T18:03:53.918ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4322024-03-21T18:03:53.919ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
4332024-03-21T18:03:53.919ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4342024-03-21T18:03:53.919ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4352024-03-21T18:03:53.923ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:36524
4362024-03-21T18:03:53.936ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4372024-03-21T18:03:53.936ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4382024-03-21T18:03:53.937ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4392024-03-21T18:03:53.938ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
4402024-03-21T18:03:53.938ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4412024-03-21T18:03:53.938ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4422024-03-21T18:03:53.955ZINFOnexus: registered new oximeter metric collection server address = [fd00:1122:3344:101::d]:12223 collector_id = c081ffa7-57b2-4200-b25f-9fd86c55e543 file = nexus/src/app/oximeter.rs:83
4432024-03-21T18:03:53.989ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = c081ffa7-57b2-4200-b25f-9fd86c55e543
4442024-03-21T18:03:53.999ZINFOnexus: populate complete; activating background tasks file = nexus/src/app/mod.rs:476
4452024-03-21T18:03:53.999ZDEBGnexus (Bootstore): roles roles = RoleSet { roles: {} }
4462024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
4472024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4482024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Signaled
4492024-03-21T18:03:53.999ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:54 reason = no blueprint
4502024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4512024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.92\u{b5}s iteration = 2
4522024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Signaled
4532024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4542024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
4552024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Signaled
4562024-03-21T18:03:53.999ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
4572024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4582024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
4592024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Signaled
4602024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Signaled
4612024-03-21T18:03:53.999ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:69 reason = no config
4622024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 9.22\u{b5}s iteration = 3
4632024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4642024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Signaled
4652024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
4662024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4672024-03-21T18:03:53.999ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:69 reason = no config
4682024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 6.85\u{b5}s iteration = 3
4692024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Signaled
4702024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
4712024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
4722024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4732024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4742024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4752024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Signaled
4762024-03-21T18:03:53.999ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
4772024-03-21T18:03:53.999ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
4782024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.42\u{b5}s iteration = 2
4792024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Signaled
4802024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4812024-03-21T18:03:53.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4822024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4832024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4842024-03-21T18:03:54.000ZDEBGnexus (Bootstore): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
4852024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Signaled
4862024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4872024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
4882024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4892024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4902024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4912024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4922024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4932024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4942024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4952024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4962024-03-21T18:03:54.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4972024-03-21T18:03:54.001ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
4982024-03-21T18:03:54.001ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.016427ms iteration = 2
4992024-03-21T18:03:54.001ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5002024-03-21T18:03:54.001ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5012024-03-21T18:03:54.001ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 729779492 } })
5022024-03-21T18:03:54.001ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 729831883 } })
5032024-03-21T18:03:54.002ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5042024-03-21T18:03:54.002ZDEBGnexus (Bootstore): roles roles = RoleSet { roles: {} }
5052024-03-21T18:03:54.002ZDEBGnexus (Bootstore): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
5062024-03-21T18:03:54.002ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5072024-03-21T18:03:54.003ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5082024-03-21T18:03:54.003ZDEBGnexus (Bootstore): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5092024-03-21T18:03:54.004ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
5102024-03-21T18:03:54.004ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 5.244914ms iteration = 2
5112024-03-21T18:03:54.004ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 732742565 } })
5122024-03-21T18:03:54.005ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
5132024-03-21T18:03:54.005ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.361214ms iteration = 2
5142024-03-21T18:03:54.005ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5152024-03-21T18:03:54.005ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5162024-03-21T18:03:54.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5172024-03-21T18:03:54.006ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
5182024-03-21T18:03:54.006ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
5192024-03-21T18:03:54.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
5202024-03-21T18:03:54.006ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5212024-03-21T18:03:54.006ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 734786982 } })
5222024-03-21T18:03:54.006ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
5232024-03-21T18:03:54.006ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5242024-03-21T18:03:54.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5252024-03-21T18:03:54.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5262024-03-21T18:03:54.007ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5272024-03-21T18:03:54.007ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
5282024-03-21T18:03:54.007ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5292024-03-21T18:03:54.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5302024-03-21T18:03:54.007ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5312024-03-21T18:03:54.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5322024-03-21T18:03:54.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5332024-03-21T18:03:54.008ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 8.185817ms iteration = 2
5342024-03-21T18:03:54.008ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
5352024-03-21T18:03:54.009ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5362024-03-21T18:03:54.009ZWARNnexus (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker file = nexus/src/app/background/sync_service_zone_nat.rs:93
5372024-03-21T18:03:54.009ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 9.613623ms iteration = 2
5382024-03-21T18:03:54.009ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1262, tv_nsec: 737242486 } })
5392024-03-21T18:03:54.025ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 101360 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::d]:36524 req_id = c7f931fc-2406-4968-a1a2-192e72283cb3 response_code = 204 uri = /metrics/collectors
5402024-03-21T18:03:54.025ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5412024-03-21T18:03:54.026ZDEBGnexus (Bootstore): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
5422024-03-21T18:03:54.026ZDEBGnexus (Bootstore): roles roles = RoleSet { roles: {} }
5432024-03-21T18:03:54.026ZDEBGnexus (Bootstore): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
5442024-03-21T18:03:54.026ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0 file = nexus/src/app/background/dns_config.rs:72
5452024-03-21T18:03:54.026ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.160686ms iteration = 2
5462024-03-21T18:03:54.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5472024-03-21T18:03:54.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5482024-03-21T18:03:54.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5492024-03-21T18:03:54.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5502024-03-21T18:03:54.027ZDEBGnexus (Bootstore): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5512024-03-21T18:03:54.027ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0 file = nexus/src/app/background/dns_config.rs:72
5522024-03-21T18:03:54.027ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 28.332988ms iteration = 2
5532024-03-21T18:03:54.028ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.547221ms iteration = 2
5542024-03-21T18:03:54.028ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5552024-03-21T18:03:54.028ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5562024-03-21T18:03:54.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5572024-03-21T18:03:54.029ZDEBGnexus (Bootstore): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
5582024-03-21T18:03:54.029ZDEBGnexus (Bootstore): roles roles = RoleSet { roles: {} }
5592024-03-21T18:03:54.029ZDEBGnexus (Bootstore): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
5602024-03-21T18:03:54.029ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone file = nexus/src/app/background/external_endpoints.rs:55
5612024-03-21T18:03:54.029ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 30.185061ms iteration = 2
5622024-03-21T18:03:54.030ZERROnexus: failed to run bootstore sync: Internal Error: no sled agents available file = nexus/src/app/mod.rs:487
5632024-03-21T18:03:54.094ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
5642024-03-21T18:03:54.094ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
5652024-03-21T18:03:54.094ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
5662024-03-21T18:03:54.095ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/ignition", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2a165cef-e358-4f5b-b2b2-ddd865327daa", "content-length": "476", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5672024-03-21T18:03:54.097ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
5682024-03-21T18:03:54.098ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "659969e5-4acd-4ebb-bf5b-a424c7871d69", "content-length": "387", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5692024-03-21T18:03:54.098ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
5702024-03-21T18:03:54.098ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5712024-03-21T18:03:54.098ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b74c3684-9895-4ed1-8160-61e19998c01d", "content-length": "26", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5722024-03-21T18:03:54.098ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5732024-03-21T18:03:54.099ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
5742024-03-21T18:03:54.100ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5752024-03-21T18:03:54.101ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/sp/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d685aa75-d51b-4fa7-aeda-d73287c8aed4", "content-length": "86", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5762024-03-21T18:03:54.101ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
5772024-03-21T18:03:54.101ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5782024-03-21T18:03:54.101ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8138660e-967a-4299-b6dc-345554cfc0de", "content-length": "26", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5792024-03-21T18:03:54.101ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5802024-03-21T18:03:54.101ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
5812024-03-21T18:03:54.101ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/sp/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "afbd32b3-b1fc-49f4-ad06-e39bfb39b47e", "content-length": "86", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5822024-03-21T18:03:54.102ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
5832024-03-21T18:03:54.102ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5842024-03-21T18:03:54.102ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f7780532-a3c7-4ae7-9254-7fd24aa2593b", "content-length": "80", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5852024-03-21T18:03:54.102ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
5862024-03-21T18:03:54.103ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "eff39a37-92af-4af4-8930-8fbb553e39d1", "content-length": "80", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5872024-03-21T18:03:54.103ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
5882024-03-21T18:03:54.104ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cmpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b756d4f1-6454-4372-a84d-92441d8aa56d", "content-length": "702", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5892024-03-21T18:03:54.105ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5902024-03-21T18:03:54.113ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f9c4b4e9-d353-4242-b0dc-c28591a05924", "content-length": "727", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5912024-03-21T18:03:54.113ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5922024-03-21T18:03:54.114ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "84553435-cfb5-4e86-b72c-637945f0f1eb", "content-length": "729", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5932024-03-21T18:03:54.114ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5942024-03-21T18:03:54.114ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "aab0652f-cb8b-44c0-93d6-2a05129fba3d", "content-length": "728", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
5952024-03-21T18:03:54.114ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
5962024-03-21T18:03:54.115ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "22400931-ee4d-4152-85ff-9a186836f569", "content-length": "375", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
5972024-03-21T18:03:54.115ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
5982024-03-21T18:03:54.115ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/sp/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3b4a239f-7230-4331-a6f5-25a39ecc32af", "content-length": "84", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
5992024-03-21T18:03:54.115ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
6002024-03-21T18:03:54.116ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/sp/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5510d897-fcd1-4794-b500-e4d25402f0ad", "content-length": "84", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
6012024-03-21T18:03:54.116ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
6022024-03-21T18:03:54.116ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fb54833b-1293-4787-97ee-adc29911c5e3", "content-length": "79", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
6032024-03-21T18:03:54.116ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
6042024-03-21T18:03:54.117ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "aca3e88b-cce3-4930-b12c-7044a27ef0b0", "content-length": "79", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
6052024-03-21T18:03:54.117ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
6062024-03-21T18:03:54.117ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cmpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6ad7ce6a-6eb5-4757-b9f1-1fbe2921cb8f", "content-length": "702", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
6072024-03-21T18:03:54.117ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6082024-03-21T18:03:54.118ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3c987d44-2161-49e6-a5aa-e001e4f660a1", "content-length": "727", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
6092024-03-21T18:03:54.118ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6102024-03-21T18:03:54.118ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d9a4dfc3-b4d3-4334-8726-615613cc6462", "content-length": "729", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
6112024-03-21T18:03:54.118ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6122024-03-21T18:03:54.119ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b7e09e17-3bf9-49a9-84e8-24f619520f88", "content-length": "728", "date": "Thu, 21 Mar 2024 18:03:54 GMT"} })
6132024-03-21T18:03:54.119ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6142024-03-21T18:03:54.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6152024-03-21T18:03:54.120ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6162024-03-21T18:03:54.122ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
6172024-03-21T18:03:54.122ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6182024-03-21T18:03:54.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6192024-03-21T18:03:54.123ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
6202024-03-21T18:03:54.123ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6212024-03-21T18:03:54.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6222024-03-21T18:03:54.124ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6232024-03-21T18:03:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
6242024-03-21T18:03:54.128ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6252024-03-21T18:03:54.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6262024-03-21T18:03:54.142ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6272024-03-21T18:03:54.142ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3d50ca15-a98e-4cf9-9d85-d3faf65b38ff", "content-length": "1", "date": "Thu, 21 Mar 2024 18:03:53 GMT"} })
6282024-03-21T18:03:54.143ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6292024-03-21T18:03:54.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6302024-03-21T18:03:54.143ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 144.236465ms iteration = 2
6312024-03-21T18:03:54.149ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 150.052609ms iteration = 2
6322024-03-21T18:03:54.308ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 45e655f1-c01d-436f-b859-c579bdf66288 file = nexus/db-queries/src/db/datastore/inventory.rs:850
6332024-03-21T18:03:54.310ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 45e655f1-c01d-436f-b859-c579bdf66288 time_started = 2024-03-21 18:03:54.094083 UTC
6342024-03-21T18:03:54.310ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 310.581306ms iteration = 2
6352024-03-21T18:03:55.525ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6362024-03-21T18:03:55.525ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6372024-03-21T18:03:55.526ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6382024-03-21T18:03:55.526ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6392024-03-21T18:03:55.527ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6402024-03-21T18:03:55.530ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
6412024-03-21T18:03:55.530ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, time_created: 2024-03-21T18:03:53.906443Z, time_modified: 2024-03-21T18:03:53.906443Z }, initialized: false, tuf_base_url: None, rack_subnet: None } file = nexus/src/app/rack.rs:647
6422024-03-21T18:03:57.531ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6432024-03-21T18:03:57.531ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6442024-03-21T18:03:57.532ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6452024-03-21T18:03:57.532ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6462024-03-21T18:03:57.538ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6472024-03-21T18:03:57.541ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
6482024-03-21T18:03:57.541ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, time_created: 2024-03-21T18:03:53.906443Z, time_modified: 2024-03-21T18:03:53.906443Z }, initialized: false, tuf_base_url: None, rack_subnet: None } file = nexus/src/app/rack.rs:647
6492024-03-21T18:03:59.542ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6502024-03-21T18:03:59.542ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6512024-03-21T18:03:59.543ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6522024-03-21T18:03:59.543ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6532024-03-21T18:03:59.544ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6542024-03-21T18:03:59.547ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
6552024-03-21T18:03:59.547ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, time_created: 2024-03-21T18:03:53.906443Z, time_modified: 2024-03-21T18:03:53.906443Z }, initialized: false, tuf_base_url: None, rack_subnet: None } file = nexus/src/app/rack.rs:647
6562024-03-21T18:04:01.548ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6572024-03-21T18:04:01.548ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6582024-03-21T18:04:01.549ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6592024-03-21T18:04:01.550ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6602024-03-21T18:04:01.550ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6612024-03-21T18:04:01.553ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
6622024-03-21T18:04:01.553ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, time_created: 2024-03-21T18:03:53.906443Z, time_modified: 2024-03-21T18:03:53.906443Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:647
6632024-03-21T18:04:03.451ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
6642024-03-21T18:04:03.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6652024-03-21T18:04:03.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6662024-03-21T18:04:03.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6672024-03-21T18:04:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6682024-03-21T18:04:03.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6692024-03-21T18:04:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6702024-03-21T18:04:03.456ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.354746ms iteration = 3
6712024-03-21T18:04:03.554ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6722024-03-21T18:04:03.555ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6732024-03-21T18:04:03.556ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6742024-03-21T18:04:03.556ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6752024-03-21T18:04:03.557ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6762024-03-21T18:04:03.559ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
6772024-03-21T18:04:03.559ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, time_created: 2024-03-21T18:03:53.906443Z, time_modified: 2024-03-21T18:03:53.906443Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:647
6782024-03-21T18:04:05.560ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6792024-03-21T18:04:05.561ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6802024-03-21T18:04:05.562ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6812024-03-21T18:04:05.562ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6822024-03-21T18:04:05.563ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6832024-03-21T18:04:05.566ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
6842024-03-21T18:04:05.566ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, time_created: 2024-03-21T18:03:53.906443Z, time_modified: 2024-03-21T18:03:53.906443Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:647
6852024-03-21T18:04:07.567ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6862024-03-21T18:04:07.567ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6872024-03-21T18:04:07.568ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6882024-03-21T18:04:07.569ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6892024-03-21T18:04:07.570ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6902024-03-21T18:04:07.572ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
6912024-03-21T18:04:07.572ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, time_created: 2024-03-21T18:03:53.906443Z, time_modified: 2024-03-21T18:03:53.906443Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:647
6922024-03-21T18:04:09.573ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6932024-03-21T18:04:09.573ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6942024-03-21T18:04:09.574ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
6952024-03-21T18:04:09.575ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
6962024-03-21T18:04:09.576ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6972024-03-21T18:04:09.578ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: 11ad3243-b088-4dcc-87e3-5a6072b1f137, lookup_type: ById(11ad3243-b088-4dcc-87e3-5a6072b1f137) } result = Ok(())
6982024-03-21T18:04:09.578ZINFOnexus: Rack initialized file = nexus/src/app/rack.rs:644
6992024-03-21T18:04:13.451ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
7002024-03-21T18:04:13.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7012024-03-21T18:04:13.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7022024-03-21T18:04:13.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7032024-03-21T18:04:13.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7042024-03-21T18:04:13.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7052024-03-21T18:04:13.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7062024-03-21T18:04:13.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7072024-03-21T18:04:13.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7082024-03-21T18:04:13.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7092024-03-21T18:04:13.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
7102024-03-21T18:04:13.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7112024-03-21T18:04:13.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7122024-03-21T18:04:13.468ZINFOnexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/blueprint_load.rs:102 target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
7132024-03-21T18:04:13.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.700089ms iteration = 4
7142024-03-21T18:04:13.468ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Dependency
7152024-03-21T18:04:13.468ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
7162024-03-21T18:04:13.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 4.029\u{b5}s iteration = 3
7172024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
7182024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
7192024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7202024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
7212024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7222024-03-21T18:04:23.451ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7232024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 3 reason = Timeout
7242024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7252024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7262024-03-21T18:04:23.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7272024-03-21T18:04:23.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7282024-03-21T18:04:23.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7292024-03-21T18:04:23.453ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
7302024-03-21T18:04:23.453ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 1.924375ms iteration = 3
7312024-03-21T18:04:23.453ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1292, tv_nsec: 181093488 } })
7322024-03-21T18:04:23.454ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1292, tv_nsec: 182361061 } })
7332024-03-21T18:04:23.454ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
7342024-03-21T18:04:23.455ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
7352024-03-21T18:04:23.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7362024-03-21T18:04:23.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7372024-03-21T18:04:23.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7382024-03-21T18:04:23.456ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
7392024-03-21T18:04:23.456ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
7402024-03-21T18:04:23.456ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
7412024-03-21T18:04:23.456ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.06\u{b5}s iteration = 3
7422024-03-21T18:04:23.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7432024-03-21T18:04:23.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7442024-03-21T18:04:23.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7452024-03-21T18:04:23.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
7462024-03-21T18:04:23.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7472024-03-21T18:04:23.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7482024-03-21T18:04:23.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
7492024-03-21T18:04:23.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.229153ms iteration = 5
7502024-03-21T18:04:23.477ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
7512024-03-21T18:04:23.477ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
7522024-03-21T18:04:23.477ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7532024-03-21T18:04:23.477ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7542024-03-21T18:04:23.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7552024-03-21T18:04:23.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7562024-03-21T18:04:23.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7572024-03-21T18:04:23.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7582024-03-21T18:04:23.481ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7592024-03-21T18:04:23.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7602024-03-21T18:04:23.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7612024-03-21T18:04:23.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7622024-03-21T18:04:23.482ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7632024-03-21T18:04:23.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7642024-03-21T18:04:23.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7652024-03-21T18:04:23.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7662024-03-21T18:04:23.484ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1292, tv_nsec: 212053513 } })
7672024-03-21T18:04:23.485ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
7682024-03-21T18:04:23.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7692024-03-21T18:04:23.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7702024-03-21T18:04:23.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7712024-03-21T18:04:23.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7722024-03-21T18:04:23.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7732024-03-21T18:04:23.505ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
7742024-03-21T18:04:23.505ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7752024-03-21T18:04:23.506ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ef7fba14-01d7-44e6-a780-7ae3fca7b124", "content-length": "26", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
7762024-03-21T18:04:23.506ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7772024-03-21T18:04:23.507ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7782024-03-21T18:04:23.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
7792024-03-21T18:04:23.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7802024-03-21T18:04:23.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7812024-03-21T18:04:23.509ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 32.31297ms iteration = 3
7822024-03-21T18:04:23.518ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
7832024-03-21T18:04:23.518ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7842024-03-21T18:04:23.518ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0c30c016-ca46-4a97-9a44-b0525bb7ea84", "content-length": "26", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
7852024-03-21T18:04:23.518ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7862024-03-21T18:04:23.519ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7872024-03-21T18:04:23.596ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
7882024-03-21T18:04:23.596ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7892024-03-21T18:04:23.597ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2892fc90-1aad-4571-8ed5-13f297985734", "content-length": "26", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
7902024-03-21T18:04:23.597ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7912024-03-21T18:04:23.597ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
7922024-03-21T18:04:23.602ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 150.560178ms iteration = 3
7932024-03-21T18:04:23.619ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7942024-03-21T18:04:23.621ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d80b55a7-6244-46b2-9873-897cde8dec08", "content-length": "1", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
7952024-03-21T18:04:23.622ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7962024-03-21T18:04:23.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7972024-03-21T18:04:23.624ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 172.687455ms iteration = 3
7982024-03-21T18:04:23.702ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7992024-03-21T18:04:23.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8002024-03-21T18:04:23.704ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
8012024-03-21T18:04:23.704ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8022024-03-21T18:04:23.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8032024-03-21T18:04:23.712ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
8042024-03-21T18:04:23.713ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8052024-03-21T18:04:23.716ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fd4e1cb0-a50d-4d0b-86b0-b8b0da4c0f99", "content-length": "121", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
8062024-03-21T18:04:23.718ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
8072024-03-21T18:04:23.718ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
8082024-03-21T18:04:23.721ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
8092024-03-21T18:04:23.721ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
8102024-03-21T18:04:23.721ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
8112024-03-21T18:04:23.721ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
8122024-03-21T18:04:23.721ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
8132024-03-21T18:04:23.721ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8142024-03-21T18:04:23.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8152024-03-21T18:04:23.723ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8162024-03-21T18:04:23.723ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c80e1237-4ea3-4973-a53b-7bd9fe356770", "content-length": "2", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
8172024-03-21T18:04:23.723ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8182024-03-21T18:04:23.724ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "70b7eab9-16f3-443a-a0df-af985f2e3440", "content-length": "52", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
8192024-03-21T18:04:23.724ZINFOnexus (BackgroundTasks): deleting loopback addresses addresses = [(Switch0, fd38:3181:a596:60e6::1)] background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
8202024-03-21T18:04:23.724ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = DELETE rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6/fd38:3181:a596:60e6::1
8212024-03-21T18:04:23.726ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6/fd38:3181:a596:60e6::1", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f5808a32-cf72-4db2-9e67-046952367012", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
8222024-03-21T18:04:23.726ZINFOnexus (BackgroundTasks): deleted loopback address address = fd38:3181:a596:60e6::1 background_task = switch_port_config_manager file = clients/dpd-client/src/lib.rs:245 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
8232024-03-21T18:04:23.726ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
8242024-03-21T18:04:23.726ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8252024-03-21T18:04:23.825ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "58f3ffb4-0a83-46ce-98e3-35e698b958b9", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
8262024-03-21T18:04:23.825ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
8272024-03-21T18:04:23.825ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8282024-03-21T18:04:23.826ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "449dc595-fdff-47d8-9909-26d6834a3790", "content-length": "443", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
8292024-03-21T18:04:23.826ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8302024-03-21T18:04:23.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8312024-03-21T18:04:23.829ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8322024-03-21T18:04:23.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8332024-03-21T18:04:23.831ZWARNnexus (BackgroundTasks): no bootstore config found in db background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:878 key = network_key rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
8342024-03-21T18:04:23.831ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8352024-03-21T18:04:23.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8362024-03-21T18:04:23.835ZINFOnexus (BackgroundTasks): updating bootstore config background_task = switch_port_config_manager config = EarlyNetworkConfig { body: EarlyNetworkConfigBody { ntp_servers: ["0.pool.ntp.org"], rack_network_config: Some(RackNetworkConfigV1 { bgp: [], infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [PortConfigV1 { addresses: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], autoneg: false, bgp_peers: [], port: "qsfp0", routes: [RouteConfig { destination: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), nexthop: 10.151.1.100 }], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G }], rack_subnet: Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 } }) }, generation: 2, schema_version: 1 } file = nexus/src/app/background/sync_switch_configuration.rs:935 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
8372024-03-21T18:04:23.835ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = PUT rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8382024-03-21T18:04:23.838ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "871bb203-90aa-4d3c-b114-c6003bc51653", "date": "Thu, 21 Mar 2024 18:04:23 GMT"} })
8392024-03-21T18:04:23.838ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8402024-03-21T18:04:23.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8412024-03-21T18:04:23.846ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 368.983633ms iteration = 3
8422024-03-21T18:04:33.451ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
8432024-03-21T18:04:33.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8442024-03-21T18:04:33.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8452024-03-21T18:04:33.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8462024-03-21T18:04:33.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8472024-03-21T18:04:33.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8482024-03-21T18:04:33.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8492024-03-21T18:04:33.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8502024-03-21T18:04:33.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8512024-03-21T18:04:33.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8522024-03-21T18:04:33.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
8532024-03-21T18:04:33.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8542024-03-21T18:04:33.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8552024-03-21T18:04:33.563ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
8562024-03-21T18:04:33.563ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 111.865145ms iteration = 6
8572024-03-21T18:04:43.451ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
8582024-03-21T18:04:43.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8592024-03-21T18:04:43.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8602024-03-21T18:04:43.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8612024-03-21T18:04:43.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8622024-03-21T18:04:43.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8632024-03-21T18:04:43.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8642024-03-21T18:04:43.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8652024-03-21T18:04:43.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8662024-03-21T18:04:43.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8672024-03-21T18:04:43.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
8682024-03-21T18:04:43.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8692024-03-21T18:04:43.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8702024-03-21T18:04:43.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
8712024-03-21T18:04:43.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.649732ms iteration = 7
8722024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
8732024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
8742024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8752024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
8762024-03-21T18:04:53.452ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
8772024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8782024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Timeout
8792024-03-21T18:04:53.452ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:69 reason = no config
8802024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
8812024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 9.791\u{b5}s iteration = 4
8822024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
8832024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
8842024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
8852024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
8862024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8872024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8882024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
8892024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8902024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 4 reason = Timeout
8912024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8922024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Timeout
8932024-03-21T18:04:53.452ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:69 reason = no config
8942024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 9.2\u{b5}s iteration = 4
8952024-03-21T18:04:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
8962024-03-21T18:04:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8972024-03-21T18:04:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8982024-03-21T18:04:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8992024-03-21T18:04:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9002024-03-21T18:04:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9012024-03-21T18:04:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9022024-03-21T18:04:53.453ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1322, tv_nsec: 180420426 } })
9032024-03-21T18:04:53.453ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1322, tv_nsec: 180586279 } })
9042024-03-21T18:04:53.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9052024-03-21T18:04:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9062024-03-21T18:04:53.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9072024-03-21T18:04:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9082024-03-21T18:04:53.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9092024-03-21T18:04:53.454ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
9102024-03-21T18:04:53.454ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.113298ms iteration = 4
9112024-03-21T18:04:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9122024-03-21T18:04:53.455ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
9132024-03-21T18:04:53.455ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.915832ms iteration = 3
9142024-03-21T18:04:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9152024-03-21T18:04:53.456ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9162024-03-21T18:04:53.456ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
9172024-03-21T18:04:53.456ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
9182024-03-21T18:04:53.456ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
9192024-03-21T18:04:53.456ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 10.32\u{b5}s iteration = 4
9202024-03-21T18:04:53.456ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
9212024-03-21T18:04:53.456ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.243166ms iteration = 3
9222024-03-21T18:04:53.458ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1322, tv_nsec: 185527628 } })
9232024-03-21T18:04:53.458ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1322, tv_nsec: 185680910 } })
9242024-03-21T18:04:53.458ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
9252024-03-21T18:04:53.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9262024-03-21T18:04:53.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9272024-03-21T18:04:53.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9282024-03-21T18:04:53.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9292024-03-21T18:04:53.459ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9302024-03-21T18:04:53.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9312024-03-21T18:04:53.460ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
9322024-03-21T18:04:53.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9332024-03-21T18:04:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9342024-03-21T18:04:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9352024-03-21T18:04:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9362024-03-21T18:04:53.461ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9372024-03-21T18:04:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9382024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9392024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
9402024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
9412024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
9422024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9432024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
9442024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
9452024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
9462024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9472024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9482024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9492024-03-21T18:04:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9502024-03-21T18:04:53.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9512024-03-21T18:04:53.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9522024-03-21T18:04:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9532024-03-21T18:04:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
9542024-03-21T18:04:53.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9552024-03-21T18:04:53.467ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9562024-03-21T18:04:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9572024-03-21T18:04:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9582024-03-21T18:04:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9592024-03-21T18:04:53.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9602024-03-21T18:04:53.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9612024-03-21T18:04:53.468ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9622024-03-21T18:04:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9632024-03-21T18:04:53.469ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
9642024-03-21T18:04:53.469ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 0 version = 2
9652024-03-21T18:04:53.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9662024-03-21T18:04:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9672024-03-21T18:04:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9682024-03-21T18:04:53.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9692024-03-21T18:04:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9702024-03-21T18:04:53.470ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9712024-03-21T18:04:53.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9722024-03-21T18:04:53.471ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
9732024-03-21T18:04:53.471ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
9742024-03-21T18:04:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9752024-03-21T18:04:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9762024-03-21T18:04:53.471ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
9772024-03-21T18:04:53.471ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.245696ms iteration = 8
9782024-03-21T18:04:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9792024-03-21T18:04:53.473ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9802024-03-21T18:04:53.473ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9812024-03-21T18:04:53.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9822024-03-21T18:04:53.474ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9832024-03-21T18:04:53.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9842024-03-21T18:04:53.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9852024-03-21T18:04:53.476ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 1 version = 2
9862024-03-21T18:04:53.476ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
9872024-03-21T18:04:53.476ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/dns_config.rs:87 generation = 2
9882024-03-21T18:04:53.476ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.501061ms iteration = 3
9892024-03-21T18:04:53.476ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Dependency
9902024-03-21T18:04:53.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9912024-03-21T18:04:53.476ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9922024-03-21T18:04:53.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9932024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
9942024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
9952024-03-21T18:04:53.478ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
9962024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9972024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.04\u{b5}s iteration = 4
9982024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
9992024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10002024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
10012024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
10022024-03-21T18:04:53.478ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/dns_config.rs:87 generation = 1
10032024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 26.26307ms iteration = 3
10042024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Dependency
10052024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10062024-03-21T18:04:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10072024-03-21T18:04:53.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10082024-03-21T18:04:53.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10092024-03-21T18:04:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10102024-03-21T18:04:53.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10112024-03-21T18:04:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10122024-03-21T18:04:53.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10132024-03-21T18:04:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10142024-03-21T18:04:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10152024-03-21T18:04:53.484ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10162024-03-21T18:04:53.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10172024-03-21T18:04:53.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1322, tv_nsec: 213387167 } })
10182024-03-21T18:04:53.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
10192024-03-21T18:04:53.493ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-21T18:03:53.561270Z, time_modified: 2024-03-21T18:03:53.561270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }) } file = nexus/src/app/background/external_endpoints.rs:84
10202024-03-21T18:04:53.493ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 41.002885ms iteration = 3
10212024-03-21T18:04:53.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10222024-03-21T18:04:53.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10232024-03-21T18:04:53.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10242024-03-21T18:04:53.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10252024-03-21T18:04:53.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10262024-03-21T18:04:53.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
10272024-03-21T18:04:53.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10282024-03-21T18:04:53.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10292024-03-21T18:04:53.509ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 30.500987ms iteration = 4
10302024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /
10312024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /access
10322024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /assets/{path}
10332024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /device/auth
10342024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /device/confirm
10352024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /device/success
10362024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /device/token
10372024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /device/verify
10382024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /experimental/v1/probes
10392024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /experimental/v1/probes
10402024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /experimental/v1/probes/{probe}
10412024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /experimental/v1/probes/{probe}
10422024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /images
10432024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /login
10442024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /login/{silo_name}/local
10452024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /login/{silo_name}/saml/{provider_name}
10462024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /login/{silo_name}/saml/{provider_name}
10472024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect
10482024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /lookup/{path}
10492024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /projects/{path}
10502024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /projects-new
10512024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /settings/{path}
10522024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /system/{path}
10532024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /utilization
10542024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/certificates
10552024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/certificates
10562024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/certificates/{certificate}
10572024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/certificates/{certificate}
10582024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/disks
10592024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks
10602024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/disks/{disk}
10612024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/disks/{disk}
10622024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks/{disk}/bulk-write
10632024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks/{disk}/bulk-write-start
10642024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks/{disk}/bulk-write-stop
10652024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/disks/{disk}/finalize
10662024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/disks/{disk}/metrics/{metric}
10672024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/floating-ips
10682024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/floating-ips
10692024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/floating-ips/{floating_ip}
10702024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/floating-ips/{floating_ip}
10712024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/floating-ips/{floating_ip}
10722024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/floating-ips/{floating_ip}/attach
10732024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/floating-ips/{floating_ip}/detach
10742024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/groups
10752024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/groups/{group_id}
10762024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/images
10772024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/images
10782024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/images/{image}
10792024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/images/{image}
10802024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/images/{image}/demote
10812024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/images/{image}/promote
10822024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances
10832024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances
10842024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/instances/{instance}
10852024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}
10862024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/disks
10872024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/disks/attach
10882024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/disks/detach
10892024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/external-ips
10902024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral
10912024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral
10922024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/migrate
10932024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/reboot
10942024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/serial-console
10952024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/serial-console/stream
10962024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/instances/{instance}/ssh-public-keys
10972024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/start
10982024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/instances/{instance}/stop
10992024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/ip-pools
11002024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/ip-pools/{pool}
11012024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/login/{silo_name}/local
11022024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/logout
11032024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/me
11042024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/me/groups
11052024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/me/ssh-keys
11062024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/me/ssh-keys
11072024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key}
11082024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/me/ssh-keys/{ssh_key}
11092024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/metrics/{metric_name}
11102024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/network-interfaces
11112024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/network-interfaces
11122024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/network-interfaces/{interface}
11132024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/network-interfaces/{interface}
11142024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/network-interfaces/{interface}
11152024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/ping
11162024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/policy
11172024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/policy
11182024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/projects
11192024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/projects
11202024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/projects/{project}
11212024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/projects/{project}
11222024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/projects/{project}
11232024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/projects/{project}/policy
11242024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/projects/{project}/policy
11252024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/snapshots
11262024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/snapshots
11272024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/snapshots/{snapshot}
11282024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/snapshots/{snapshot}
11292024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/disks
11302024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/racks
11312024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/racks/{rack_id}
11322024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds
11332024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/hardware/sleds
11342024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds/{sled_id}
11352024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks
11362024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances
11372024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy
11382024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/sleds-uninitialized
11392024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/switch-port
11402024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings
11412024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings
11422024-03-21T18:04:53.514ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/switches
11432024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/hardware/switches/{switch_id}
11442024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/identity-providers
11452024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/identity-providers/local/users
11462024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id}
11472024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password
11482024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/identity-providers/saml
11492024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/identity-providers/saml/{provider}
11502024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools
11512024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools
11522024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/ip-pools/{pool}
11532024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools/{pool}
11542024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/ip-pools/{pool}
11552024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools/{pool}/ranges
11562024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add
11572024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove
11582024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools/{pool}/silos
11592024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools/{pool}/silos
11602024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo}
11612024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo}
11622024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools/{pool}/utilization
11632024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools-service
11642024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/ip-pools-service/ranges
11652024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools-service/ranges/add
11662024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/ip-pools-service/ranges/remove
11672024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/metrics/{metric_name}
11682024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/address-lot
11692024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/address-lot
11702024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot}
11712024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks
11722024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/bfd-disable
11732024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/bfd-enable
11742024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bfd-status
11752024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/bgp
11762024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp
11772024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/bgp
11782024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/bgp-announce
11792024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-announce
11802024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/bgp-announce
11812024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-message-history
11822024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-routes-ipv4
11832024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/bgp-status
11842024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/loopback-address
11852024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/loopback-address
11862024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask}
11872024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/networking/switch-port-settings
11882024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/switch-port-settings
11892024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/networking/switch-port-settings
11902024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/networking/switch-port-settings/{port}
11912024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/policy
11922024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/policy
11932024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/roles
11942024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/roles/{role_name}
11952024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silo-quotas
11962024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos
11972024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/system/silos
11982024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/system/silos/{silo}
11992024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos/{silo}
12002024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos/{silo}/ip-pools
12012024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos/{silo}/policy
12022024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/silos/{silo}/policy
12032024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/silos/{silo}/quotas
12042024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/silos/{silo}/quotas
12052024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/system/update/repository
12062024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/update/repository/{system_version}
12072024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/users
12082024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/users/{user_id}
12092024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/users-builtin
12102024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/users-builtin/{user}
12112024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/utilization/silos
12122024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/system/utilization/silos/{silo}
12132024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/users
12142024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/utilization
12152024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-firewall-rules
12162024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpc-firewall-rules
12172024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-router-routes
12182024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/vpc-router-routes
12192024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/vpc-router-routes/{route}
12202024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-router-routes/{route}
12212024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpc-router-routes/{route}
12222024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-routers
12232024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/vpc-routers
12242024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/vpc-routers/{router}
12252024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-routers/{router}
12262024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpc-routers/{router}
12272024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-subnets
12282024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/vpc-subnets
12292024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/vpc-subnets/{subnet}
12302024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-subnets/{subnet}
12312024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpc-subnets/{subnet}
12322024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces
12332024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpcs
12342024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = POST path = /v1/vpcs
12352024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = DELETE path = /v1/vpcs/{vpc}
12362024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = GET path = /v1/vpcs/{vpc}
12372024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.6:443 method = PUT path = /v1/vpcs/{vpc}
12382024-03-21T18:04:53.515ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:195 local_addr = 172.30.2.6:443
12392024-03-21T18:04:53.515ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.6:443
12402024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /
12412024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /access
12422024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /assets/{path}
12432024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /device/auth
12442024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /device/confirm
12452024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /device/success
12462024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /device/token
12472024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /device/verify
12482024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /experimental/v1/probes
12492024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /experimental/v1/probes
12502024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /experimental/v1/probes/{probe}
12512024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /experimental/v1/probes/{probe}
12522024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /images
12532024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /login
12542024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /login/{silo_name}/local
12552024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}
12562024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /login/{silo_name}/saml/{provider_name}
12572024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect
12582024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /lookup/{path}
12592024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /projects/{path}
12602024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /projects-new
12612024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /settings/{path}
12622024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /system/{path}
12632024-03-21T18:04:53.521ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /utilization
12642024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/certificates
12652024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/certificates
12662024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/certificates/{certificate}
12672024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/certificates/{certificate}
12682024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/disks
12692024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks
12702024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/disks/{disk}
12712024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/disks/{disk}
12722024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks/{disk}/bulk-write
12732024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start
12742024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop
12752024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/disks/{disk}/finalize
12762024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/disks/{disk}/metrics/{metric}
12772024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/floating-ips
12782024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/floating-ips
12792024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/floating-ips/{floating_ip}
12802024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/floating-ips/{floating_ip}
12812024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/floating-ips/{floating_ip}
12822024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach
12832024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach
12842024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/groups
12852024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/groups/{group_id}
12862024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/images
12872024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/images
12882024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/images/{image}
12892024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/images/{image}
12902024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/images/{image}/demote
12912024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/images/{image}/promote
12922024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances
12932024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances
12942024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/instances/{instance}
12952024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}
12962024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/disks
12972024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/disks/attach
12982024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/disks/detach
12992024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/external-ips
13002024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral
13012024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral
13022024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/migrate
13032024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/reboot
13042024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/serial-console
13052024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream
13062024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys
13072024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/start
13082024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/instances/{instance}/stop
13092024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/ip-pools
13102024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/ip-pools/{pool}
13112024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/login/{silo_name}/local
13122024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/logout
13132024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/me
13142024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/me/groups
13152024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/me/ssh-keys
13162024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/me/ssh-keys
13172024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key}
13182024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key}
13192024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/metrics/{metric_name}
13202024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/network-interfaces
13212024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/network-interfaces
13222024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/network-interfaces/{interface}
13232024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/network-interfaces/{interface}
13242024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/network-interfaces/{interface}
13252024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/ping
13262024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/policy
13272024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/policy
13282024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/projects
13292024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/projects
13302024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/projects/{project}
13312024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/projects/{project}
13322024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/projects/{project}
13332024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/projects/{project}/policy
13342024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/projects/{project}/policy
13352024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/snapshots
13362024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/snapshots
13372024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/snapshots/{snapshot}
13382024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/snapshots/{snapshot}
13392024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/disks
13402024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/racks
13412024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/racks/{rack_id}
13422024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds
13432024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/hardware/sleds
13442024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}
13452024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks
13462024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances
13472024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy
13482024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized
13492024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/switch-port
13502024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings
13512024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings
13522024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/switches
13532024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/hardware/switches/{switch_id}
13542024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/identity-providers
13552024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/identity-providers/local/users
13562024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id}
13572024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password
13582024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/identity-providers/saml
13592024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/identity-providers/saml/{provider}
13602024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools
13612024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools
13622024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/ip-pools/{pool}
13632024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools/{pool}
13642024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/ip-pools/{pool}
13652024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges
13662024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add
13672024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove
13682024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos
13692024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos
13702024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo}
13712024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo}
13722024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization
13732024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools-service
13742024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/ip-pools-service/ranges
13752024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add
13762024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove
13772024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/metrics/{metric_name}
13782024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/address-lot
13792024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/address-lot
13802024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot}
13812024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks
13822024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/bfd-disable
13832024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/bfd-enable
13842024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bfd-status
13852024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/bgp
13862024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp
13872024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/bgp
13882024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/bgp-announce
13892024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-announce
13902024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/bgp-announce
13912024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-message-history
13922024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4
13932024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/bgp-status
13942024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/loopback-address
13952024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/loopback-address
13962024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask}
13972024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/networking/switch-port-settings
13982024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/switch-port-settings
13992024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/networking/switch-port-settings
14002024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port}
14012024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/policy
14022024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/policy
14032024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/roles
14042024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/roles/{role_name}
14052024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silo-quotas
14062024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos
14072024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/system/silos
14082024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/system/silos/{silo}
14092024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos/{silo}
14102024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools
14112024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos/{silo}/policy
14122024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/silos/{silo}/policy
14132024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/silos/{silo}/quotas
14142024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/silos/{silo}/quotas
14152024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/system/update/repository
14162024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/update/repository/{system_version}
14172024-03-21T18:04:53.522ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/users
14182024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/users/{user_id}
14192024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/users-builtin
14202024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/users-builtin/{user}
14212024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/utilization/silos
14222024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/system/utilization/silos/{silo}
14232024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/users
14242024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/utilization
14252024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-firewall-rules
14262024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpc-firewall-rules
14272024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-router-routes
14282024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/vpc-router-routes
14292024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/vpc-router-routes/{route}
14302024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-router-routes/{route}
14312024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpc-router-routes/{route}
14322024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-routers
14332024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/vpc-routers
14342024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/vpc-routers/{router}
14352024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-routers/{router}
14362024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpc-routers/{router}
14372024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-subnets
14382024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/vpc-subnets
14392024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/vpc-subnets/{subnet}
14402024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-subnets/{subnet}
14412024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpc-subnets/{subnet}
14422024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces
14432024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpcs
14442024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = POST path = /v1/vpcs
14452024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = DELETE path = /v1/vpcs/{vpc}
14462024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = GET path = /v1/vpcs/{vpc}
14472024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::b]:12228 method = PUT path = /v1/vpcs/{vpc}
14482024-03-21T18:04:53.523ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:195 local_addr = [fd00:1122:3344:101::b]:12228
14492024-03-21T18:04:53.523ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::b]:12228
14502024-03-21T18:04:53.523ZDEBGnexus: registering nexus as metric producer
14512024-03-21T18:04:53.586ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
14522024-03-21T18:04:53.586ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14532024-03-21T18:04:53.586ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cbf52749-abc6-4db8-953b-37a788a73b51", "content-length": "26", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
14542024-03-21T18:04:53.586ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14552024-03-21T18:04:53.588ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14562024-03-21T18:04:53.647ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
14572024-03-21T18:04:53.647ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14582024-03-21T18:04:53.647ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fd06cbac-17bf-4158-9caf-ab1616ea78de", "content-length": "26", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
14592024-03-21T18:04:53.648ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14602024-03-21T18:04:53.649ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
14612024-03-21T18:04:53.663ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
14622024-03-21T18:04:53.763ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
14632024-03-21T18:04:53.763ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14642024-03-21T18:04:53.765ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "033e9c14-66c1-4674-a5af-41e53fdc0316", "content-length": "26", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
14652024-03-21T18:04:53.765ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14662024-03-21T18:04:53.765ZDEBGnexus: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
14672024-03-21T18:04:53.766ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:56732
14682024-03-21T18:04:53.766ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14692024-03-21T18:04:53.844ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
14702024-03-21T18:04:53.882ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
14712024-03-21T18:04:53.930ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 477.991037ms iteration = 4
14722024-03-21T18:04:53.982ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
14732024-03-21T18:04:53.982ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "47663f7e-a0a2-433b-8479-0ac453bfd515", "content-length": "1", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
14742024-03-21T18:04:53.983ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14752024-03-21T18:04:53.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14762024-03-21T18:04:53.985ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 532.512454ms iteration = 4
14772024-03-21T18:04:54.034ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
14782024-03-21T18:04:54.035ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5d9f975a-ee9b-43e8-bc88-288222b9516e", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14792024-03-21T18:04:54.035ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14802024-03-21T18:04:54.036ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6eb93f9d-15d5-437e-a5b6-c8aae0fb8fa9", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14812024-03-21T18:04:54.036ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14822024-03-21T18:04:54.037ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d7c62173-b5c4-413a-a440-3dfdfe547814", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14832024-03-21T18:04:54.037ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14842024-03-21T18:04:54.039ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
14852024-03-21T18:04:54.039ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 560.242572ms iteration = 5
14862024-03-21T18:04:54.084ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = c081ffa7-57b2-4200-b25f-9fd86c55e543
14872024-03-21T18:04:54.092ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = c081ffa7-57b2-4200-b25f-9fd86c55e543 uri = http://[fd00:1122:3344:101::d]:12223/producers
14882024-03-21T18:04:54.092ZDEBGnexus: client response oximeter-collector = c081ffa7-57b2-4200-b25f-9fd86c55e543 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c693a609-7e3a-4056-a3e7-4a061eab38b1", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
14892024-03-21T18:04:54.092ZINFOnexus: assigned collector to new producer collector_id = c081ffa7-57b2-4200-b25f-9fd86c55e543 file = nexus/src/app/oximeter.rs:193 producer_id = 63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
14902024-03-21T18:04:54.094ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 328511 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:56732 req_id = 2a3dc8f4-27e4-44db-8151-67008ef6a6ac response_code = 204 uri = /metrics/producers
14912024-03-21T18:04:54.094ZDEBGnexus: client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2a3dc8f4-27e4-44db-8151-67008ef6a6ac", "date": "Thu, 21 Mar 2024 18:04:54 GMT"} })
14922024-03-21T18:04:54.095ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
14932024-03-21T18:04:54.095ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3843858b-9521-4368-bd8e-8fe55a305e27", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14942024-03-21T18:04:54.095ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14952024-03-21T18:04:54.097ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "74fd7fd4-2c52-4f8d-ac6f-12d5ab09a145", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14962024-03-21T18:04:54.097ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14972024-03-21T18:04:54.099ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14982024-03-21T18:04:54.099ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 622.982725ms iteration = 5
14992024-03-21T18:04:54.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15002024-03-21T18:04:54.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15012024-03-21T18:04:54.102ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
15022024-03-21T18:04:54.102ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15032024-03-21T18:04:54.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15042024-03-21T18:04:54.107ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
15052024-03-21T18:04:54.107ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15062024-03-21T18:04:54.108ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "47f1663e-92dd-43de-8797-6c8e3a31345f", "content-length": "121", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
15072024-03-21T18:04:54.108ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
15082024-03-21T18:04:54.108ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
15092024-03-21T18:04:54.109ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
15102024-03-21T18:04:54.109ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
15112024-03-21T18:04:54.109ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
15122024-03-21T18:04:54.109ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
15132024-03-21T18:04:54.109ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
15142024-03-21T18:04:54.109ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15152024-03-21T18:04:54.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15162024-03-21T18:04:54.110ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15172024-03-21T18:04:54.110ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c0f8f6f1-d241-430e-994c-6d12f83b37f6", "content-length": "2", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
15182024-03-21T18:04:54.110ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15192024-03-21T18:04:54.110ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fab0687d-c97f-45eb-8d23-d73fb9e03ef9", "content-length": "2", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
15202024-03-21T18:04:54.110ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
15212024-03-21T18:04:54.110ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15222024-03-21T18:04:54.187ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1c6acc61-7151-4519-85c4-3e8dfc4e00de", "date": "Thu, 21 Mar 2024 18:04:54 GMT"} })
15232024-03-21T18:04:54.187ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
15242024-03-21T18:04:54.187ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15252024-03-21T18:04:54.187ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bf374a18-8bbc-43a8-a794-c35f48169b58", "content-length": "443", "date": "Thu, 21 Mar 2024 18:04:53 GMT"} })
15262024-03-21T18:04:54.187ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15272024-03-21T18:04:54.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15282024-03-21T18:04:54.191ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15292024-03-21T18:04:54.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15302024-03-21T18:04:54.201ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 723.066119ms iteration = 4
15312024-03-21T18:05:03.451ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
15322024-03-21T18:05:03.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15332024-03-21T18:05:03.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15342024-03-21T18:05:03.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15352024-03-21T18:05:03.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15362024-03-21T18:05:03.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15372024-03-21T18:05:03.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15382024-03-21T18:05:03.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15392024-03-21T18:05:03.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15402024-03-21T18:05:03.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15412024-03-21T18:05:03.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
15422024-03-21T18:05:03.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15432024-03-21T18:05:03.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15442024-03-21T18:05:03.472ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
15452024-03-21T18:05:03.472ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.65507ms iteration = 9
15462024-03-21T18:05:04.124ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:43220
15472024-03-21T18:05:04.131ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 6345 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 66c0c70b-e72d-4fb8-acca-38b6d707d3f7 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
15482024-03-21T18:05:13.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
15492024-03-21T18:05:13.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15502024-03-21T18:05:13.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15512024-03-21T18:05:13.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15522024-03-21T18:05:13.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15532024-03-21T18:05:13.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15542024-03-21T18:05:13.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15552024-03-21T18:05:13.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15562024-03-21T18:05:13.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15572024-03-21T18:05:13.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15582024-03-21T18:05:13.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
15592024-03-21T18:05:13.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15602024-03-21T18:05:13.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15612024-03-21T18:05:13.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
15622024-03-21T18:05:13.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.592571ms iteration = 10
15632024-03-21T18:05:14.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 936 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = d07329c6-b833-46cf-ab2d-e77347b7e3a3 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
15642024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
15652024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
15662024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15672024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15682024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
15692024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15702024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
15712024-03-21T18:05:23.452ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
15722024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
15732024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15742024-03-21T18:05:23.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15752024-03-21T18:05:23.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15762024-03-21T18:05:23.454ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1352, tv_nsec: 181519376 } })
15772024-03-21T18:05:23.454ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
15782024-03-21T18:05:23.454ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.335592ms iteration = 5
15792024-03-21T18:05:23.455ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
15802024-03-21T18:05:23.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15812024-03-21T18:05:23.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15822024-03-21T18:05:23.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15832024-03-21T18:05:23.457ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
15842024-03-21T18:05:23.457ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
15852024-03-21T18:05:23.457ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
15862024-03-21T18:05:23.457ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.43\u{b5}s iteration = 5
15872024-03-21T18:05:23.457ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1352, tv_nsec: 184220204 } })
15882024-03-21T18:05:23.457ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
15892024-03-21T18:05:23.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15902024-03-21T18:05:23.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15912024-03-21T18:05:23.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15922024-03-21T18:05:23.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
15932024-03-21T18:05:23.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15942024-03-21T18:05:23.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15952024-03-21T18:05:23.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
15962024-03-21T18:05:23.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.838104ms iteration = 11
15972024-03-21T18:05:23.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
15982024-03-21T18:05:23.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
15992024-03-21T18:05:23.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16002024-03-21T18:05:23.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16012024-03-21T18:05:23.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16022024-03-21T18:05:23.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16032024-03-21T18:05:23.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16042024-03-21T18:05:23.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16052024-03-21T18:05:23.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
16062024-03-21T18:05:23.482ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16072024-03-21T18:05:23.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16082024-03-21T18:05:23.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16092024-03-21T18:05:23.482ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16102024-03-21T18:05:23.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16112024-03-21T18:05:23.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16122024-03-21T18:05:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16132024-03-21T18:05:23.484ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1352, tv_nsec: 211522473 } })
16142024-03-21T18:05:23.485ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
16152024-03-21T18:05:23.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16162024-03-21T18:05:23.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16172024-03-21T18:05:23.500ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16182024-03-21T18:05:23.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16192024-03-21T18:05:23.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16202024-03-21T18:05:23.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
16212024-03-21T18:05:23.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16222024-03-21T18:05:23.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16232024-03-21T18:05:23.504ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.688018ms iteration = 5
16242024-03-21T18:05:23.537ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
16252024-03-21T18:05:23.537ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16262024-03-21T18:05:23.537ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4db7214b-7312-47a9-90c2-f17789252f01", "content-length": "26", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16272024-03-21T18:05:23.537ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16282024-03-21T18:05:23.539ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16292024-03-21T18:05:23.544ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
16302024-03-21T18:05:23.544ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16312024-03-21T18:05:23.545ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8ef41688-a3c7-4ec6-b1ac-a462929672d0", "content-length": "26", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16322024-03-21T18:05:23.545ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16332024-03-21T18:05:23.547ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16342024-03-21T18:05:23.578ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
16352024-03-21T18:05:23.578ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16362024-03-21T18:05:23.579ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bce4b9f8-4422-4e84-b170-cbedf31bcaa7", "content-length": "26", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16372024-03-21T18:05:23.579ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16382024-03-21T18:05:23.580ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
16392024-03-21T18:05:23.627ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
16402024-03-21T18:05:23.628ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "09fd9767-6344-477c-bcd8-6056359714d2", "content-length": "1", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16412024-03-21T18:05:23.628ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16422024-03-21T18:05:23.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16432024-03-21T18:05:23.630ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 177.835997ms iteration = 5
16442024-03-21T18:05:23.636ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 184.201241ms iteration = 5
16452024-03-21T18:05:23.706ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16462024-03-21T18:05:23.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16472024-03-21T18:05:23.708ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
16482024-03-21T18:05:23.708ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16492024-03-21T18:05:23.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16502024-03-21T18:05:23.713ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
16512024-03-21T18:05:23.713ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
16522024-03-21T18:05:23.714ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "39d910e5-9c80-4c53-963e-66d72f19f036", "content-length": "121", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16532024-03-21T18:05:23.714ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
16542024-03-21T18:05:23.714ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
16552024-03-21T18:05:23.714ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
16562024-03-21T18:05:23.714ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
16572024-03-21T18:05:23.714ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
16582024-03-21T18:05:23.714ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
16592024-03-21T18:05:23.714ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
16602024-03-21T18:05:23.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16612024-03-21T18:05:23.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16622024-03-21T18:05:23.716ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
16632024-03-21T18:05:23.716ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b165e8f8-5b45-4be2-ab73-e138f9f938b1", "content-length": "2", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16642024-03-21T18:05:23.716ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
16652024-03-21T18:05:23.716ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "75496046-69d7-40ce-9014-28338314da74", "content-length": "52", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16662024-03-21T18:05:23.716ZINFOnexus (BackgroundTasks): deleting loopback addresses addresses = [(Switch0, fd38:3181:a596:60e6::1)] background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
16672024-03-21T18:05:23.716ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = DELETE rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6/fd38:3181:a596:60e6::1
16682024-03-21T18:05:23.717ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6/fd38:3181:a596:60e6::1", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0a5a9f27-6d29-4ca3-8d7b-730931e62757", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16692024-03-21T18:05:23.717ZINFOnexus (BackgroundTasks): deleted loopback address address = fd38:3181:a596:60e6::1 background_task = switch_port_config_manager file = clients/dpd-client/src/lib.rs:245 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
16702024-03-21T18:05:23.717ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
16712024-03-21T18:05:23.717ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
16722024-03-21T18:05:23.766ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "806d5882-c5e8-47ac-bcac-795930b56d5a", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16732024-03-21T18:05:23.766ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
16742024-03-21T18:05:23.766ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
16752024-03-21T18:05:23.766ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "59a9925a-36be-4479-9983-4be6a06c5892", "content-length": "443", "date": "Thu, 21 Mar 2024 18:05:23 GMT"} })
16762024-03-21T18:05:23.766ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16772024-03-21T18:05:23.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16782024-03-21T18:05:23.769ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16792024-03-21T18:05:23.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16802024-03-21T18:05:23.775ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 297.466731ms iteration = 5
16812024-03-21T18:05:24.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 247 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 4e3a28c2-12db-4de9-8e95-4c2ded5f375e response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
16822024-03-21T18:05:33.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
16832024-03-21T18:05:33.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16842024-03-21T18:05:33.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16852024-03-21T18:05:33.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16862024-03-21T18:05:33.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16872024-03-21T18:05:33.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16882024-03-21T18:05:33.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16892024-03-21T18:05:33.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16902024-03-21T18:05:33.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16912024-03-21T18:05:33.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16922024-03-21T18:05:33.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
16932024-03-21T18:05:33.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16942024-03-21T18:05:33.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16952024-03-21T18:05:33.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
16962024-03-21T18:05:33.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.355617ms iteration = 12
16972024-03-21T18:05:34.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 249 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 520f26a8-0da0-4572-ad6b-59a0473e60e2 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
16982024-03-21T18:05:43.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
16992024-03-21T18:05:43.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17002024-03-21T18:05:43.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17012024-03-21T18:05:43.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17022024-03-21T18:05:43.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17032024-03-21T18:05:43.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17042024-03-21T18:05:43.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17052024-03-21T18:05:43.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17062024-03-21T18:05:43.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17072024-03-21T18:05:43.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17082024-03-21T18:05:43.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
17092024-03-21T18:05:43.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17102024-03-21T18:05:43.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17112024-03-21T18:05:43.465ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
17122024-03-21T18:05:43.465ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 12.673197ms iteration = 13
17132024-03-21T18:05:44.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 169 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 0dd398ea-f542-4a13-805b-a177710d15f1 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
17142024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
17152024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17162024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
17172024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
17182024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
17192024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17202024-03-21T18:05:53.453ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
17212024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
17222024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
17232024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
17242024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
17252024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17262024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
17272024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
17282024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17292024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
17302024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17312024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
17322024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
17332024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
17342024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17352024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17362024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17372024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17382024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17392024-03-21T18:05:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
17402024-03-21T18:05:53.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17412024-03-21T18:05:53.454ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17422024-03-21T18:05:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17432024-03-21T18:05:53.455ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1382, tv_nsec: 181648888 } })
17442024-03-21T18:05:53.455ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1382, tv_nsec: 181667458 } })
17452024-03-21T18:05:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17462024-03-21T18:05:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17472024-03-21T18:05:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17482024-03-21T18:05:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17492024-03-21T18:05:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17502024-03-21T18:05:53.457ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
17512024-03-21T18:05:53.457ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
17522024-03-21T18:05:53.457ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
17532024-03-21T18:05:53.457ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.16\u{b5}s iteration = 6
17542024-03-21T18:05:53.458ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
17552024-03-21T18:05:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.898538ms iteration = 4
17562024-03-21T18:05:53.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17572024-03-21T18:05:53.459ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17582024-03-21T18:05:53.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17592024-03-21T18:05:53.459ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
17602024-03-21T18:05:53.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.68819ms iteration = 4
17612024-03-21T18:05:53.460ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1382, tv_nsec: 186568676 } })
17622024-03-21T18:05:53.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17632024-03-21T18:05:53.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17642024-03-21T18:05:53.460ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1382, tv_nsec: 186585877 } })
17652024-03-21T18:05:53.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17662024-03-21T18:05:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17672024-03-21T18:05:53.461ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
17682024-03-21T18:05:53.461ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
17692024-03-21T18:05:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17702024-03-21T18:05:53.465ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17712024-03-21T18:05:53.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17722024-03-21T18:05:53.466ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
17732024-03-21T18:05:53.466ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
17742024-03-21T18:05:53.466ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
17752024-03-21T18:05:53.466ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17762024-03-21T18:05:53.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17772024-03-21T18:05:53.471ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
17782024-03-21T18:05:53.471ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
17792024-03-21T18:05:53.471ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
17802024-03-21T18:05:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17812024-03-21T18:05:53.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17822024-03-21T18:05:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17832024-03-21T18:05:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17842024-03-21T18:05:53.471ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17852024-03-21T18:05:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17862024-03-21T18:05:53.472ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
17872024-03-21T18:05:53.472ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 18.97891ms iteration = 6
17882024-03-21T18:05:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17892024-03-21T18:05:53.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17902024-03-21T18:05:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17912024-03-21T18:05:53.477ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17922024-03-21T18:05:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17932024-03-21T18:05:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
17942024-03-21T18:05:53.477ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17952024-03-21T18:05:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
17962024-03-21T18:05:53.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17972024-03-21T18:05:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17982024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17992024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18002024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18012024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18022024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
18032024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
18042024-03-21T18:05:53.478ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
18052024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.33\u{b5}s iteration = 5
18062024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
18072024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18082024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18092024-03-21T18:05:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18102024-03-21T18:05:53.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18112024-03-21T18:05:53.479ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
18122024-03-21T18:05:53.479ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
18132024-03-21T18:05:53.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18142024-03-21T18:05:53.479ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18152024-03-21T18:05:53.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18162024-03-21T18:05:53.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18172024-03-21T18:05:53.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18182024-03-21T18:05:53.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18192024-03-21T18:05:53.480ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
18202024-03-21T18:05:53.480ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 0 version = 2
18212024-03-21T18:05:53.480ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18222024-03-21T18:05:53.480ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18232024-03-21T18:05:53.480ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18242024-03-21T18:05:53.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18252024-03-21T18:05:53.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18262024-03-21T18:05:53.482ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
18272024-03-21T18:05:53.482ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.457458ms iteration = 14
18282024-03-21T18:05:53.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
18292024-03-21T18:05:53.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18302024-03-21T18:05:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18312024-03-21T18:05:53.485ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
18322024-03-21T18:05:53.485ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18332024-03-21T18:05:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18342024-03-21T18:05:53.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
18352024-03-21T18:05:53.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18362024-03-21T18:05:53.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18372024-03-21T18:05:53.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18382024-03-21T18:05:53.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18392024-03-21T18:05:53.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18402024-03-21T18:05:53.487ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18412024-03-21T18:05:53.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18422024-03-21T18:05:53.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18432024-03-21T18:05:53.487ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18442024-03-21T18:05:53.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18452024-03-21T18:05:53.488ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1382, tv_nsec: 214972315 } })
18462024-03-21T18:05:53.488ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
18472024-03-21T18:05:53.489ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
18482024-03-21T18:05:53.489ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
18492024-03-21T18:05:53.489ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-21 18:04:07.863813 UTC dns_group = internal generation = 1
18502024-03-21T18:05:53.489ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 35.821062ms iteration = 4
18512024-03-21T18:05:53.489ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 1 version = 2
18522024-03-21T18:05:53.489ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
18532024-03-21T18:05:53.489ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-21 18:04:08.259984 UTC dns_group = external generation = 2
18542024-03-21T18:05:53.489ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 36.602576ms iteration = 4
18552024-03-21T18:05:53.494ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-21T18:03:53.561270Z, time_modified: 2024-03-21T18:03:53.561270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }) }
18562024-03-21T18:05:53.494ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 41.3075ms iteration = 4
18572024-03-21T18:05:53.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18582024-03-21T18:05:53.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18592024-03-21T18:05:53.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18602024-03-21T18:05:53.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18612024-03-21T18:05:53.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18622024-03-21T18:05:53.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
18632024-03-21T18:05:53.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18642024-03-21T18:05:53.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18652024-03-21T18:05:53.510ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.486904ms iteration = 6
18662024-03-21T18:05:53.651ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
18672024-03-21T18:05:53.651ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18682024-03-21T18:05:53.651ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e804b5c1-a343-4d20-9787-9d1971ffd202", "content-length": "26", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
18692024-03-21T18:05:53.651ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18702024-03-21T18:05:53.652ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18712024-03-21T18:05:53.715ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
18722024-03-21T18:05:53.728ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
18732024-03-21T18:05:53.735ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
18742024-03-21T18:05:53.735ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18752024-03-21T18:05:53.737ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d5210eb9-8c29-417e-97cf-c23246395f04", "content-length": "26", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
18762024-03-21T18:05:53.737ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18772024-03-21T18:05:53.738ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18782024-03-21T18:05:53.771ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
18792024-03-21T18:05:53.771ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18802024-03-21T18:05:53.771ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "686ebdf1-11d3-4f8d-bb51-e64719a9f0b7", "content-length": "26", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
18812024-03-21T18:05:53.771ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18822024-03-21T18:05:53.773ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
18832024-03-21T18:05:53.920ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
18842024-03-21T18:05:53.920ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5593d825-d1bc-47de-a538-0695125efb35", "content-length": "1", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
18852024-03-21T18:05:53.921ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18862024-03-21T18:05:53.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18872024-03-21T18:05:53.923ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 470.413741ms iteration = 6
18882024-03-21T18:05:53.976ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
18892024-03-21T18:05:54.013ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
18902024-03-21T18:05:54.013ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "df93794b-d44c-497a-a8fc-9aff85b260f9", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18912024-03-21T18:05:54.013ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18922024-03-21T18:05:54.015ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "39d653c1-07c2-404e-8171-4d9ffc1f0798", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18932024-03-21T18:05:54.015ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18942024-03-21T18:05:54.016ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18952024-03-21T18:05:54.016ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 563.023291ms iteration = 6
18962024-03-21T18:05:54.058ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 605.278998ms iteration = 6
18972024-03-21T18:05:54.084ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
18982024-03-21T18:05:54.085ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "42ff10fa-710b-4fd3-aa19-9194a27f2148", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
18992024-03-21T18:05:54.085ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
19002024-03-21T18:05:54.086ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ca2b91e1-0a38-4158-a4b4-a51cc0b13161", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
19012024-03-21T18:05:54.086ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
19022024-03-21T18:05:54.087ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "97cb7e9b-dc75-4caa-9d64-56729b02db52", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
19032024-03-21T18:05:54.087ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
19042024-03-21T18:05:54.089ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
19052024-03-21T18:05:54.089ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 636.015359ms iteration = 6
19062024-03-21T18:05:54.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 244 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 1ffe76fe-6abb-4a82-b8c0-e1864f9c55c2 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
19072024-03-21T18:05:54.131ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19082024-03-21T18:05:54.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19092024-03-21T18:05:54.133ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
19102024-03-21T18:05:54.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19112024-03-21T18:05:54.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19122024-03-21T18:05:54.141ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
19132024-03-21T18:05:54.141ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
19142024-03-21T18:05:54.142ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ffebaa26-cf25-429f-9d32-ce4a9827b2a8", "content-length": "121", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
19152024-03-21T18:05:54.142ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
19162024-03-21T18:05:54.142ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
19172024-03-21T18:05:54.142ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
19182024-03-21T18:05:54.142ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
19192024-03-21T18:05:54.142ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
19202024-03-21T18:05:54.142ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
19212024-03-21T18:05:54.142ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
19222024-03-21T18:05:54.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19232024-03-21T18:05:54.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19242024-03-21T18:05:54.144ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
19252024-03-21T18:05:54.144ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a111750d-585b-4a33-b7c7-9deb7db7684a", "content-length": "2", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
19262024-03-21T18:05:54.144ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
19272024-03-21T18:05:54.144ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cb155d23-b3d0-4416-9e1a-10aaaa10e3c1", "content-length": "2", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
19282024-03-21T18:05:54.144ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
19292024-03-21T18:05:54.144ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
19302024-03-21T18:05:54.190ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ce14c6fa-e954-43a2-bd70-847bf75374fc", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
19312024-03-21T18:05:54.190ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
19322024-03-21T18:05:54.190ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
19332024-03-21T18:05:54.190ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "83c37a28-02ee-498c-9bc6-fc53a6be62d3", "content-length": "443", "date": "Thu, 21 Mar 2024 18:05:53 GMT"} })
19342024-03-21T18:05:54.190ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19352024-03-21T18:05:54.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19362024-03-21T18:05:54.193ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19372024-03-21T18:05:54.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19382024-03-21T18:05:54.200ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 721.976949ms iteration = 6
19392024-03-21T18:06:03.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
19402024-03-21T18:06:03.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19412024-03-21T18:06:03.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19422024-03-21T18:06:03.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19432024-03-21T18:06:03.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19442024-03-21T18:06:03.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19452024-03-21T18:06:03.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19462024-03-21T18:06:03.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19472024-03-21T18:06:03.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19482024-03-21T18:06:03.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19492024-03-21T18:06:03.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
19502024-03-21T18:06:03.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19512024-03-21T18:06:03.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19522024-03-21T18:06:03.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
19532024-03-21T18:06:03.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.737286ms iteration = 15
19542024-03-21T18:06:04.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 156 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 4b14cf32-dadd-43bf-950f-9edc3ad8a7a2 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
19552024-03-21T18:06:13.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
19562024-03-21T18:06:13.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19572024-03-21T18:06:13.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19582024-03-21T18:06:13.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19592024-03-21T18:06:13.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19602024-03-21T18:06:13.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19612024-03-21T18:06:13.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19622024-03-21T18:06:13.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19632024-03-21T18:06:13.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19642024-03-21T18:06:13.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19652024-03-21T18:06:13.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
19662024-03-21T18:06:13.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19672024-03-21T18:06:13.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19682024-03-21T18:06:13.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
19692024-03-21T18:06:13.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.38036ms iteration = 16
19702024-03-21T18:06:14.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 257 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 022f611f-4749-4efa-b072-560c0b1c1cc1 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
19712024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
19722024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
19732024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19742024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19752024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
19762024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19772024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
19782024-03-21T18:06:23.453ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
19792024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19802024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19812024-03-21T18:06:23.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19822024-03-21T18:06:23.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19832024-03-21T18:06:23.456ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
19842024-03-21T18:06:23.456ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.886072ms iteration = 7
19852024-03-21T18:06:23.456ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1412, tv_nsec: 182667516 } })
19862024-03-21T18:06:23.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
19872024-03-21T18:06:23.457ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
19882024-03-21T18:06:23.457ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
19892024-03-21T18:06:23.457ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
19902024-03-21T18:06:23.457ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.37\u{b5}s iteration = 7
19912024-03-21T18:06:23.457ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1412, tv_nsec: 184046081 } })
19922024-03-21T18:06:23.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19932024-03-21T18:06:23.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19942024-03-21T18:06:23.458ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
19952024-03-21T18:06:23.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19962024-03-21T18:06:23.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19972024-03-21T18:06:23.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19982024-03-21T18:06:23.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19992024-03-21T18:06:23.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
20002024-03-21T18:06:23.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20012024-03-21T18:06:23.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20022024-03-21T18:06:23.470ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
20032024-03-21T18:06:23.470ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.227949ms iteration = 17
20042024-03-21T18:06:23.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
20052024-03-21T18:06:23.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
20062024-03-21T18:06:23.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20072024-03-21T18:06:23.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20082024-03-21T18:06:23.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20092024-03-21T18:06:23.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20102024-03-21T18:06:23.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20112024-03-21T18:06:23.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20122024-03-21T18:06:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20132024-03-21T18:06:23.484ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20142024-03-21T18:06:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20152024-03-21T18:06:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20162024-03-21T18:06:23.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20172024-03-21T18:06:23.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20182024-03-21T18:06:23.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20192024-03-21T18:06:23.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20202024-03-21T18:06:23.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1412, tv_nsec: 213609251 } })
20212024-03-21T18:06:23.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
20222024-03-21T18:06:23.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20232024-03-21T18:06:23.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20242024-03-21T18:06:23.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20252024-03-21T18:06:23.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20262024-03-21T18:06:23.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20272024-03-21T18:06:23.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
20282024-03-21T18:06:23.510ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20292024-03-21T18:06:23.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20302024-03-21T18:06:23.511ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 32.519873ms iteration = 7
20312024-03-21T18:06:23.537ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
20322024-03-21T18:06:23.537ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20332024-03-21T18:06:23.537ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "66282ed3-2fe2-44ec-9ad3-c85d4c3c1799", "content-length": "26", "date": "Thu, 21 Mar 2024 18:06:23 GMT"} })
20342024-03-21T18:06:23.537ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20352024-03-21T18:06:23.539ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20362024-03-21T18:06:23.610ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
20372024-03-21T18:06:23.610ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20382024-03-21T18:06:23.610ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9625768d-1586-411d-b84d-fceb51b754ed", "content-length": "26", "date": "Thu, 21 Mar 2024 18:06:23 GMT"} })
20392024-03-21T18:06:23.610ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20402024-03-21T18:06:23.612ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20412024-03-21T18:06:23.640ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
20422024-03-21T18:06:23.641ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f0476d62-20de-4196-b38a-d24342cf610d", "content-length": "1", "date": "Thu, 21 Mar 2024 18:06:22 GMT"} })
20432024-03-21T18:06:23.642ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20442024-03-21T18:06:23.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20452024-03-21T18:06:23.643ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 190.684818ms iteration = 7
20462024-03-21T18:06:23.673ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
20472024-03-21T18:06:23.673ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20482024-03-21T18:06:23.673ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fc4f248b-1f1e-4d1c-8774-6f092ee6e143", "content-length": "26", "date": "Thu, 21 Mar 2024 18:06:23 GMT"} })
20492024-03-21T18:06:23.673ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20502024-03-21T18:06:23.675ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
20512024-03-21T18:06:23.746ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 293.597862ms iteration = 7
20522024-03-21T18:06:23.817ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20532024-03-21T18:06:23.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20542024-03-21T18:06:23.819ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
20552024-03-21T18:06:23.819ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20562024-03-21T18:06:23.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20572024-03-21T18:06:23.825ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
20582024-03-21T18:06:23.826ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
20592024-03-21T18:06:23.826ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a7706472-89d1-4daf-9f2a-e24c6c12d81d", "content-length": "121", "date": "Thu, 21 Mar 2024 18:06:23 GMT"} })
20602024-03-21T18:06:23.826ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
20612024-03-21T18:06:23.826ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
20622024-03-21T18:06:23.826ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
20632024-03-21T18:06:23.826ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
20642024-03-21T18:06:23.826ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
20652024-03-21T18:06:23.826ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
20662024-03-21T18:06:23.826ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
20672024-03-21T18:06:23.826ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20682024-03-21T18:06:23.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20692024-03-21T18:06:23.828ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
20702024-03-21T18:06:23.828ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "96c824fb-c247-4b3c-905a-0405f60b8e25", "content-length": "2", "date": "Thu, 21 Mar 2024 18:06:23 GMT"} })
20712024-03-21T18:06:23.828ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
20722024-03-21T18:06:23.828ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d3b0a808-8428-4df8-9bcb-f0dd85a63e74", "content-length": "2", "date": "Thu, 21 Mar 2024 18:06:23 GMT"} })
20732024-03-21T18:06:23.828ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
20742024-03-21T18:06:23.828ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20752024-03-21T18:06:23.873ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b4f5d512-d874-4983-9a11-1e6ca32b4a69", "date": "Thu, 21 Mar 2024 18:06:23 GMT"} })
20762024-03-21T18:06:23.874ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
20772024-03-21T18:06:23.874ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20782024-03-21T18:06:23.874ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "18742a3c-1b41-4b61-9f0b-4a460a34db2c", "content-length": "443", "date": "Thu, 21 Mar 2024 18:06:23 GMT"} })
20792024-03-21T18:06:23.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20802024-03-21T18:06:23.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20812024-03-21T18:06:23.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20822024-03-21T18:06:23.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20832024-03-21T18:06:23.883ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 404.381017ms iteration = 7
20842024-03-21T18:06:24.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 137 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 65c3cffa-c018-40f1-a80b-9c70cb76a89c response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
20852024-03-21T18:06:33.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
20862024-03-21T18:06:33.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20872024-03-21T18:06:33.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20882024-03-21T18:06:33.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20892024-03-21T18:06:33.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20902024-03-21T18:06:33.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20912024-03-21T18:06:33.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20922024-03-21T18:06:33.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20932024-03-21T18:06:33.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20942024-03-21T18:06:33.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20952024-03-21T18:06:33.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
20962024-03-21T18:06:33.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20972024-03-21T18:06:33.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20982024-03-21T18:06:33.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
20992024-03-21T18:06:33.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.626394ms iteration = 18
21002024-03-21T18:06:34.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 148 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = a7dc1051-f6ed-45a5-a787-27ff7b1243de response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
21012024-03-21T18:06:43.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
21022024-03-21T18:06:43.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21032024-03-21T18:06:43.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21042024-03-21T18:06:43.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21052024-03-21T18:06:43.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21062024-03-21T18:06:43.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21072024-03-21T18:06:43.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21082024-03-21T18:06:43.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21092024-03-21T18:06:43.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21102024-03-21T18:06:43.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21112024-03-21T18:06:43.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
21122024-03-21T18:06:43.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21132024-03-21T18:06:43.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21142024-03-21T18:06:43.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
21152024-03-21T18:06:43.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.566234ms iteration = 19
21162024-03-21T18:06:44.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 167 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = b83ead87-1deb-4c83-a053-925a9036c6a5 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
21172024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
21182024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21192024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
21202024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
21212024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21222024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
21232024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
21242024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
21252024-03-21T18:06:53.452ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
21262024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21272024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
21282024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
21292024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
21302024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
21312024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21322024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
21332024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
21342024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21352024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
21362024-03-21T18:06:53.452ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21372024-03-21T18:06:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21382024-03-21T18:06:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21392024-03-21T18:06:53.453ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1442, tv_nsec: 179430918 } })
21402024-03-21T18:06:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21412024-03-21T18:06:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21422024-03-21T18:06:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
21432024-03-21T18:06:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21442024-03-21T18:06:53.453ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1442, tv_nsec: 179590081 } })
21452024-03-21T18:06:53.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21462024-03-21T18:06:53.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21472024-03-21T18:06:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21482024-03-21T18:06:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21492024-03-21T18:06:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21502024-03-21T18:06:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21512024-03-21T18:06:53.455ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21522024-03-21T18:06:53.456ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
21532024-03-21T18:06:53.456ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.35009ms iteration = 8
21542024-03-21T18:06:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21552024-03-21T18:06:53.456ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
21562024-03-21T18:06:53.457ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.254466ms iteration = 5
21572024-03-21T18:06:53.457ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
21582024-03-21T18:06:53.457ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
21592024-03-21T18:06:53.457ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
21602024-03-21T18:06:53.457ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 11.3\u{b5}s iteration = 8
21612024-03-21T18:06:53.458ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1442, tv_nsec: 184424628 } })
21622024-03-21T18:06:53.458ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
21632024-03-21T18:06:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 5.59365ms iteration = 5
21642024-03-21T18:06:53.458ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1442, tv_nsec: 184612302 } })
21652024-03-21T18:06:53.459ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
21662024-03-21T18:06:53.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21672024-03-21T18:06:53.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21682024-03-21T18:06:53.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21692024-03-21T18:06:53.459ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21702024-03-21T18:06:53.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21712024-03-21T18:06:53.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21722024-03-21T18:06:53.460ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
21732024-03-21T18:06:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21742024-03-21T18:06:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21752024-03-21T18:06:53.461ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21762024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21772024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21782024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21792024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
21802024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
21812024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
21822024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21832024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21842024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
21852024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
21862024-03-21T18:06:53.462ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
21872024-03-21T18:06:53.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21882024-03-21T18:06:53.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21892024-03-21T18:06:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21902024-03-21T18:06:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21912024-03-21T18:06:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21922024-03-21T18:06:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21932024-03-21T18:06:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
21942024-03-21T18:06:53.467ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21952024-03-21T18:06:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21962024-03-21T18:06:53.468ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21972024-03-21T18:06:53.468ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21982024-03-21T18:06:53.468ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
21992024-03-21T18:06:53.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22002024-03-21T18:06:53.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22012024-03-21T18:06:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22022024-03-21T18:06:53.469ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22032024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22042024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22052024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22062024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22072024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22082024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 0 version = 2
22092024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22102024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22112024-03-21T18:06:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22122024-03-21T18:06:53.471ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
22132024-03-21T18:06:53.471ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22142024-03-21T18:06:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22152024-03-21T18:06:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22162024-03-21T18:06:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22172024-03-21T18:06:53.472ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22182024-03-21T18:06:53.472ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22192024-03-21T18:06:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22202024-03-21T18:06:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22212024-03-21T18:06:53.473ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
22222024-03-21T18:06:53.473ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.779173ms iteration = 20
22232024-03-21T18:06:53.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22242024-03-21T18:06:53.476ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22252024-03-21T18:06:53.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22262024-03-21T18:06:53.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22272024-03-21T18:06:53.476ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22282024-03-21T18:06:53.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22292024-03-21T18:06:53.477ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 1 version = 2
22302024-03-21T18:06:53.477ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
22312024-03-21T18:06:53.477ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-21 18:04:08.259984 UTC dns_group = external generation = 2
22322024-03-21T18:06:53.477ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.828285ms iteration = 5
22332024-03-21T18:06:53.478ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
22342024-03-21T18:06:53.478ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22352024-03-21T18:06:53.478ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-21 18:04:07.863813 UTC dns_group = internal generation = 1
22362024-03-21T18:06:53.478ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 26.091297ms iteration = 5
22372024-03-21T18:06:53.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
22382024-03-21T18:06:53.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
22392024-03-21T18:06:53.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22402024-03-21T18:06:53.479ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
22412024-03-21T18:06:53.479ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 5.95\u{b5}s iteration = 6
22422024-03-21T18:06:53.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
22432024-03-21T18:06:53.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22442024-03-21T18:06:53.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22452024-03-21T18:06:53.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22462024-03-21T18:06:53.480ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-21T18:03:53.561270Z, time_modified: 2024-03-21T18:03:53.561270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }) }
22472024-03-21T18:06:53.480ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 27.584264ms iteration = 5
22482024-03-21T18:06:53.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22492024-03-21T18:06:53.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22502024-03-21T18:06:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22512024-03-21T18:06:53.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22522024-03-21T18:06:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22532024-03-21T18:06:53.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22542024-03-21T18:06:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22552024-03-21T18:06:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22562024-03-21T18:06:53.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22572024-03-21T18:06:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22582024-03-21T18:06:53.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1442, tv_nsec: 212172605 } })
22592024-03-21T18:06:53.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
22602024-03-21T18:06:53.498ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22612024-03-21T18:06:53.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22622024-03-21T18:06:53.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22632024-03-21T18:06:53.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22642024-03-21T18:06:53.500ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22652024-03-21T18:06:53.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
22662024-03-21T18:06:53.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22672024-03-21T18:06:53.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22682024-03-21T18:06:53.504ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.299913ms iteration = 8
22692024-03-21T18:06:53.610ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
22702024-03-21T18:06:53.610ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22712024-03-21T18:06:53.610ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "412f27bf-7002-4bde-81f5-6df956c68006", "content-length": "26", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} })
22722024-03-21T18:06:53.610ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22732024-03-21T18:06:53.612ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22742024-03-21T18:06:53.721ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
22752024-03-21T18:06:53.723ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
22762024-03-21T18:06:53.735ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
22772024-03-21T18:06:53.735ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22782024-03-21T18:06:53.735ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "97ef4f78-280e-4408-a278-e04a9a9c95c1", "content-length": "26", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} })
22792024-03-21T18:06:53.735ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22802024-03-21T18:06:53.737ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22812024-03-21T18:06:53.777ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
22822024-03-21T18:06:53.777ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22832024-03-21T18:06:53.777ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f157bd0c-c09d-4925-a509-0776ca1901ea", "content-length": "26", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} })
22842024-03-21T18:06:53.777ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22852024-03-21T18:06:53.779ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
22862024-03-21T18:06:53.792ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 339.578056ms iteration = 8
22872024-03-21T18:06:53.881ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
22882024-03-21T18:06:53.884ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
22892024-03-21T18:06:53.884ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "47bf6a7e-4669-4f71-b336-152dde7edc9f", "content-length": "1", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} })
22902024-03-21T18:06:53.884ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22912024-03-21T18:06:53.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22922024-03-21T18:06:53.886ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 433.651482ms iteration = 8
22932024-03-21T18:06:53.927ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
22942024-03-21T18:06:53.928ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "276527b7-225a-46fe-a35f-f79e215c4538", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22952024-03-21T18:06:53.928ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22962024-03-21T18:06:53.928ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5e46e2b0-4e8d-42a4-b0d0-531a87bca3dd", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22972024-03-21T18:06:53.928ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22982024-03-21T18:06:53.930ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22992024-03-21T18:06:53.930ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 477.261164ms iteration = 7
23002024-03-21T18:06:53.964ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
23012024-03-21T18:06:53.965ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3a2efb64-0559-48d3-9233-120bbe27a82f", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23022024-03-21T18:06:53.965ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23032024-03-21T18:06:53.966ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4ed79517-973b-4a4b-a5a1-056827999598", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23042024-03-21T18:06:53.966ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23052024-03-21T18:06:53.968ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "27f0dbed-92a6-43ac-b466-0b9427ceaebb", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23062024-03-21T18:06:53.968ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23072024-03-21T18:06:53.969ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23082024-03-21T18:06:53.970ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 517.22684ms iteration = 7
23092024-03-21T18:06:54.017ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23102024-03-21T18:06:54.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23112024-03-21T18:06:54.020ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
23122024-03-21T18:06:54.020ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23132024-03-21T18:06:54.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23142024-03-21T18:06:54.028ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
23152024-03-21T18:06:54.028ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
23162024-03-21T18:06:54.028ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8d706548-6faa-4da4-8dbf-0318420a7779", "content-length": "121", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} })
23172024-03-21T18:06:54.028ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
23182024-03-21T18:06:54.029ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
23192024-03-21T18:06:54.029ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
23202024-03-21T18:06:54.029ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
23212024-03-21T18:06:54.029ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
23222024-03-21T18:06:54.029ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
23232024-03-21T18:06:54.029ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
23242024-03-21T18:06:54.029ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23252024-03-21T18:06:54.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23262024-03-21T18:06:54.031ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
23272024-03-21T18:06:54.031ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b93e61b1-3a11-42f8-b118-892570f224d5", "content-length": "2", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} })
23282024-03-21T18:06:54.031ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
23292024-03-21T18:06:54.031ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bf8396db-c09e-4b4e-8e31-33cad12810bd", "content-length": "2", "date": "Thu, 21 Mar 2024 18:06:53 GMT"} })
23302024-03-21T18:06:54.031ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
23312024-03-21T18:06:54.031ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23322024-03-21T18:06:54.105ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a42ad731-bfb5-45f8-86bb-a4857fe2f035", "date": "Thu, 21 Mar 2024 18:06:54 GMT"} })
23332024-03-21T18:06:54.105ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
23342024-03-21T18:06:54.105ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23352024-03-21T18:06:54.106ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "65d9ed13-cc25-4dfb-9020-488c5fe4da7e", "content-length": "443", "date": "Thu, 21 Mar 2024 18:06:54 GMT"} })
23362024-03-21T18:06:54.106ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23372024-03-21T18:06:54.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23382024-03-21T18:06:54.109ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23392024-03-21T18:06:54.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23402024-03-21T18:06:54.115ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 636.380805ms iteration = 8
23412024-03-21T18:06:54.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 184 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = a4cca8f0-7fbc-41f0-8717-e097f7aacfef response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
23422024-03-21T18:07:03.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
23432024-03-21T18:07:03.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23442024-03-21T18:07:03.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23452024-03-21T18:07:03.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23462024-03-21T18:07:03.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23472024-03-21T18:07:03.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23482024-03-21T18:07:03.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23492024-03-21T18:07:03.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23502024-03-21T18:07:03.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23512024-03-21T18:07:03.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23522024-03-21T18:07:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
23532024-03-21T18:07:03.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23542024-03-21T18:07:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23552024-03-21T18:07:03.470ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
23562024-03-21T18:07:03.470ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.226129ms iteration = 21
23572024-03-21T18:07:04.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 857 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = ac1c1512-7f50-451a-a795-0410352a2425 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
23582024-03-21T18:07:13.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
23592024-03-21T18:07:13.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23602024-03-21T18:07:13.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23612024-03-21T18:07:13.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23622024-03-21T18:07:13.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23632024-03-21T18:07:13.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23642024-03-21T18:07:13.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23652024-03-21T18:07:13.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23662024-03-21T18:07:13.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23672024-03-21T18:07:13.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23682024-03-21T18:07:13.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
23692024-03-21T18:07:13.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23702024-03-21T18:07:13.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23712024-03-21T18:07:13.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
23722024-03-21T18:07:13.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.96739ms iteration = 22
23732024-03-21T18:07:14.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 161 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 050713ef-2188-411d-9525-18fe8f6e06cf response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
23742024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
23752024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23762024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
23772024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
23782024-03-21T18:07:23.453ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
23792024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
23802024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
23812024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23822024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23832024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23842024-03-21T18:07:23.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
23852024-03-21T18:07:23.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23862024-03-21T18:07:23.455ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
23872024-03-21T18:07:23.455ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.25569ms iteration = 9
23882024-03-21T18:07:23.455ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1472, tv_nsec: 181566447 } })
23892024-03-21T18:07:23.455ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1472, tv_nsec: 181760260 } })
23902024-03-21T18:07:23.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
23912024-03-21T18:07:23.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23922024-03-21T18:07:23.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23932024-03-21T18:07:23.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23942024-03-21T18:07:23.457ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
23952024-03-21T18:07:23.458ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
23962024-03-21T18:07:23.458ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
23972024-03-21T18:07:23.458ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
23982024-03-21T18:07:23.458ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.42\u{b5}s iteration = 9
23992024-03-21T18:07:23.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24002024-03-21T18:07:23.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24012024-03-21T18:07:23.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24022024-03-21T18:07:23.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
24032024-03-21T18:07:23.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24042024-03-21T18:07:23.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24052024-03-21T18:07:23.469ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
24062024-03-21T18:07:23.469ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.084849ms iteration = 23
24072024-03-21T18:07:23.478ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
24082024-03-21T18:07:23.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24092024-03-21T18:07:23.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
24102024-03-21T18:07:23.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24112024-03-21T18:07:23.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24122024-03-21T18:07:23.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24132024-03-21T18:07:23.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24142024-03-21T18:07:23.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24152024-03-21T18:07:23.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
24162024-03-21T18:07:23.482ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24172024-03-21T18:07:23.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24182024-03-21T18:07:23.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
24192024-03-21T18:07:23.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24202024-03-21T18:07:23.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24212024-03-21T18:07:23.485ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1472, tv_nsec: 211230228 } })
24222024-03-21T18:07:23.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24232024-03-21T18:07:23.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24242024-03-21T18:07:23.486ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
24252024-03-21T18:07:23.498ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24262024-03-21T18:07:23.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24272024-03-21T18:07:23.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24282024-03-21T18:07:23.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24292024-03-21T18:07:23.500ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
24302024-03-21T18:07:23.500ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24312024-03-21T18:07:23.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24322024-03-21T18:07:23.502ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2efe6373-e49e-4b61-9843-e9134bef2f29", "content-length": "26", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24332024-03-21T18:07:23.502ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24342024-03-21T18:07:23.502ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
24352024-03-21T18:07:23.502ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24362024-03-21T18:07:23.503ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2dd2f61f-d966-4ee4-81e5-03844bdf35a5", "content-length": "26", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24372024-03-21T18:07:23.503ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24382024-03-21T18:07:23.503ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24392024-03-21T18:07:23.504ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24402024-03-21T18:07:23.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
24412024-03-21T18:07:23.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24422024-03-21T18:07:23.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24432024-03-21T18:07:23.505ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.866561ms iteration = 9
24442024-03-21T18:07:23.612ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
24452024-03-21T18:07:23.613ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ca4fd9a-7a1e-4818-b7a2-238ef33ee461", "content-length": "1", "date": "Thu, 21 Mar 2024 18:07:22 GMT"} })
24462024-03-21T18:07:23.614ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24472024-03-21T18:07:23.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24482024-03-21T18:07:23.615ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 162.779228ms iteration = 9
24492024-03-21T18:07:23.624ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
24502024-03-21T18:07:23.624ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24512024-03-21T18:07:23.624ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "542f4ce4-c3b5-4828-8fa8-740b62287fe9", "content-length": "26", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24522024-03-21T18:07:23.624ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24532024-03-21T18:07:23.625ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
24542024-03-21T18:07:23.651ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 198.807713ms iteration = 9
24552024-03-21T18:07:23.726ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24562024-03-21T18:07:23.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24572024-03-21T18:07:23.728ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
24582024-03-21T18:07:23.728ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24592024-03-21T18:07:23.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24602024-03-21T18:07:23.734ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
24612024-03-21T18:07:23.734ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
24622024-03-21T18:07:23.734ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0cc8bfe9-6a03-4dc9-8b90-a49275b79165", "content-length": "121", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24632024-03-21T18:07:23.734ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
24642024-03-21T18:07:23.734ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
24652024-03-21T18:07:23.735ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
24662024-03-21T18:07:23.735ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
24672024-03-21T18:07:23.735ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
24682024-03-21T18:07:23.735ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
24692024-03-21T18:07:23.735ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
24702024-03-21T18:07:23.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24712024-03-21T18:07:23.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24722024-03-21T18:07:23.739ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
24732024-03-21T18:07:23.739ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d38efffe-47fd-44bf-b8cd-9a99dbb8d93f", "content-length": "2", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24742024-03-21T18:07:23.739ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
24752024-03-21T18:07:23.739ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "73a4f57c-e562-4700-a763-f4a8ed8b702f", "content-length": "52", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24762024-03-21T18:07:23.739ZINFOnexus (BackgroundTasks): deleting loopback addresses addresses = [(Switch0, fd38:3181:a596:60e6::1)] background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
24772024-03-21T18:07:23.739ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = DELETE rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6/fd38:3181:a596:60e6::1
24782024-03-21T18:07:23.740ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6/fd38:3181:a596:60e6::1", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4a5fad06-88e2-4df2-8eb9-7179384a8bbf", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24792024-03-21T18:07:23.740ZINFOnexus (BackgroundTasks): deleted loopback address address = fd38:3181:a596:60e6::1 background_task = switch_port_config_manager file = clients/dpd-client/src/lib.rs:245 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
24802024-03-21T18:07:23.740ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
24812024-03-21T18:07:23.740ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24822024-03-21T18:07:23.792ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1cc26a2a-3193-4737-8e7f-008844312680", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24832024-03-21T18:07:23.792ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
24842024-03-21T18:07:23.792ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
24852024-03-21T18:07:23.792ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6d9a8562-bd53-4d95-b89c-79c816ab94a1", "content-length": "443", "date": "Thu, 21 Mar 2024 18:07:23 GMT"} })
24862024-03-21T18:07:23.792ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24872024-03-21T18:07:23.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24882024-03-21T18:07:23.795ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24892024-03-21T18:07:23.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24902024-03-21T18:07:23.801ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 322.643173ms iteration = 9
24912024-03-21T18:07:24.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 170 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = c33b1cff-3f6f-423f-9ea4-aa552c2166f6 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
24922024-03-21T18:07:33.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
24932024-03-21T18:07:33.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24942024-03-21T18:07:33.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24952024-03-21T18:07:33.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24962024-03-21T18:07:33.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24972024-03-21T18:07:33.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24982024-03-21T18:07:33.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24992024-03-21T18:07:33.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25002024-03-21T18:07:33.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25012024-03-21T18:07:33.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25022024-03-21T18:07:33.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
25032024-03-21T18:07:33.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25042024-03-21T18:07:33.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25052024-03-21T18:07:33.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
25062024-03-21T18:07:33.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.664032ms iteration = 24
25072024-03-21T18:07:34.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 165 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = a88b2e1f-494b-4a55-87b9-981a1706558c response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
25082024-03-21T18:07:43.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
25092024-03-21T18:07:43.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25102024-03-21T18:07:43.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25112024-03-21T18:07:43.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25122024-03-21T18:07:43.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25132024-03-21T18:07:43.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25142024-03-21T18:07:43.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25152024-03-21T18:07:43.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25162024-03-21T18:07:43.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25172024-03-21T18:07:43.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25182024-03-21T18:07:43.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
25192024-03-21T18:07:43.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25202024-03-21T18:07:43.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25212024-03-21T18:07:43.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
25222024-03-21T18:07:43.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.304226ms iteration = 25
25232024-03-21T18:07:44.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 164 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 952962e7-ace1-47f2-9a19-24b291ecd907 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
25242024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
25252024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
25262024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25272024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25282024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
25292024-03-21T18:07:53.453ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
25302024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
25312024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
25322024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25332024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
25342024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
25352024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
25362024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
25372024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
25382024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25392024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
25402024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
25412024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25422024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
25432024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
25442024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25452024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25462024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25472024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25482024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25492024-03-21T18:07:53.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
25502024-03-21T18:07:53.453ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 179582771 } })
25512024-03-21T18:07:53.453ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 179679113 } })
25522024-03-21T18:07:53.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25532024-03-21T18:07:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25542024-03-21T18:07:53.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25552024-03-21T18:07:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25562024-03-21T18:07:53.455ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25572024-03-21T18:07:53.455ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
25582024-03-21T18:07:53.455ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.593117ms iteration = 10
25592024-03-21T18:07:53.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25602024-03-21T18:07:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25612024-03-21T18:07:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25622024-03-21T18:07:53.457ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
25632024-03-21T18:07:53.457ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.785388ms iteration = 6
25642024-03-21T18:07:53.458ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
25652024-03-21T18:07:53.458ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
25662024-03-21T18:07:53.458ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
25672024-03-21T18:07:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.631\u{b5}s iteration = 10
25682024-03-21T18:07:53.458ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
25692024-03-21T18:07:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.102091ms iteration = 6
25702024-03-21T18:07:53.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
25712024-03-21T18:07:53.458ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 184335926 } })
25722024-03-21T18:07:53.458ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 184339376 } })
25732024-03-21T18:07:53.458ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25742024-03-21T18:07:53.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25752024-03-21T18:07:53.459ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
25762024-03-21T18:07:53.460ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25772024-03-21T18:07:53.460ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
25782024-03-21T18:07:53.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25792024-03-21T18:07:53.461ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25802024-03-21T18:07:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25812024-03-21T18:07:53.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25822024-03-21T18:07:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25832024-03-21T18:07:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25842024-03-21T18:07:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25852024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
25862024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
25872024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
25882024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
25892024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
25902024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
25912024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25922024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25932024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25942024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25952024-03-21T18:07:53.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25962024-03-21T18:07:53.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25972024-03-21T18:07:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25982024-03-21T18:07:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25992024-03-21T18:07:53.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26002024-03-21T18:07:53.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
26012024-03-21T18:07:53.465ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26022024-03-21T18:07:53.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26032024-03-21T18:07:53.467ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26042024-03-21T18:07:53.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26052024-03-21T18:07:53.469ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26062024-03-21T18:07:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26072024-03-21T18:07:53.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26082024-03-21T18:07:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26092024-03-21T18:07:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
26102024-03-21T18:07:53.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26112024-03-21T18:07:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26122024-03-21T18:07:53.471ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
26132024-03-21T18:07:53.471ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
26142024-03-21T18:07:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26152024-03-21T18:07:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
26162024-03-21T18:07:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26172024-03-21T18:07:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26182024-03-21T18:07:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26192024-03-21T18:07:53.471ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
26202024-03-21T18:07:53.472ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26212024-03-21T18:07:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26222024-03-21T18:07:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26232024-03-21T18:07:53.473ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
26242024-03-21T18:07:53.473ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 0 version = 2
26252024-03-21T18:07:53.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26262024-03-21T18:07:53.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26272024-03-21T18:07:53.474ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26282024-03-21T18:07:53.475ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
26292024-03-21T18:07:53.475ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.27036ms iteration = 26
26302024-03-21T18:07:53.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26312024-03-21T18:07:53.476ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26322024-03-21T18:07:53.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26332024-03-21T18:07:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
26342024-03-21T18:07:53.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26352024-03-21T18:07:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26362024-03-21T18:07:53.478ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
26372024-03-21T18:07:53.478ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
26382024-03-21T18:07:53.478ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-21 18:04:07.863813 UTC dns_group = internal generation = 1
26392024-03-21T18:07:53.478ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 25.057239ms iteration = 6
26402024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
26412024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
26422024-03-21T18:07:53.479ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
26432024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 4.32\u{b5}s iteration = 7
26442024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26452024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
26462024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26472024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26482024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26492024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 1 version = 2
26502024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
26512024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-21 18:04:08.259984 UTC dns_group = external generation = 2
26522024-03-21T18:07:53.479ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 26.472725ms iteration = 6
26532024-03-21T18:07:53.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26542024-03-21T18:07:53.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26552024-03-21T18:07:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26562024-03-21T18:07:53.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26572024-03-21T18:07:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
26582024-03-21T18:07:53.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26592024-03-21T18:07:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26602024-03-21T18:07:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26612024-03-21T18:07:53.484ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-21T18:03:53.561270Z, time_modified: 2024-03-21T18:03:53.561270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }) }
26622024-03-21T18:07:53.484ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 31.578136ms iteration = 6
26632024-03-21T18:07:53.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26642024-03-21T18:07:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26652024-03-21T18:07:53.485ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 211694747 } })
26662024-03-21T18:07:53.486ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
26672024-03-21T18:07:53.498ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26682024-03-21T18:07:53.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26692024-03-21T18:07:53.500ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26702024-03-21T18:07:53.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26712024-03-21T18:07:53.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26722024-03-21T18:07:53.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
26732024-03-21T18:07:53.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26742024-03-21T18:07:53.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26752024-03-21T18:07:53.505ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.485955ms iteration = 10
26762024-03-21T18:07:53.615ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
26772024-03-21T18:07:53.724ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
26782024-03-21T18:07:53.727ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
26792024-03-21T18:07:53.727ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26802024-03-21T18:07:53.728ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "223dbb6c-5cce-41c8-a6bb-70cec5a54716", "content-length": "26", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} })
26812024-03-21T18:07:53.728ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26822024-03-21T18:07:53.729ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26832024-03-21T18:07:53.733ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
26842024-03-21T18:07:53.733ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26852024-03-21T18:07:53.733ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fe108090-39a1-481d-98c6-0230c8303a80", "content-length": "26", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} })
26862024-03-21T18:07:53.733ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26872024-03-21T18:07:53.734ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26882024-03-21T18:07:53.756ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
26892024-03-21T18:07:53.756ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26902024-03-21T18:07:53.757ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a5bac38f-be8e-44d5-b8b2-909d1e6cc164", "content-length": "26", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} })
26912024-03-21T18:07:53.757ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26922024-03-21T18:07:53.759ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
26932024-03-21T18:07:53.805ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
26942024-03-21T18:07:53.895ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 442.049652ms iteration = 10
26952024-03-21T18:07:53.898ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
26962024-03-21T18:07:53.898ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "51ffcdba-7a6a-4a42-ac18-1a9761585f13", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26972024-03-21T18:07:53.898ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26982024-03-21T18:07:53.900ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "89328b51-b08f-4f83-8323-5e66b5c70e0e", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26992024-03-21T18:07:53.900ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
27002024-03-21T18:07:53.901ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
27012024-03-21T18:07:53.901ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 448.369166ms iteration = 8
27022024-03-21T18:07:53.978ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
27032024-03-21T18:07:53.978ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4b8718e3-5486-4007-9c02-e6b7d3dd996c", "content-length": "1", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} })
27042024-03-21T18:07:53.980ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27052024-03-21T18:07:53.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27062024-03-21T18:07:53.982ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 528.843229ms iteration = 10
27072024-03-21T18:07:53.998ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
27082024-03-21T18:07:53.998ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "dfa762be-d635-44f3-b6dc-a2fcd1beb095", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27092024-03-21T18:07:53.998ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27102024-03-21T18:07:54.001ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4e8b8569-87e5-44d6-b7dc-7d8564776e92", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27112024-03-21T18:07:54.001ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27122024-03-21T18:07:54.003ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "13abe6b0-094a-4d4a-ba17-ba844ee08d89", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27132024-03-21T18:07:54.003ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27142024-03-21T18:07:54.005ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27152024-03-21T18:07:54.005ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 551.675067ms iteration = 8
27162024-03-21T18:07:54.026ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27172024-03-21T18:07:54.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27182024-03-21T18:07:54.029ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
27192024-03-21T18:07:54.029ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27202024-03-21T18:07:54.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27212024-03-21T18:07:54.036ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
27222024-03-21T18:07:54.036ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
27232024-03-21T18:07:54.037ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "85c1b553-cb47-4d36-85d2-d0241213251d", "content-length": "121", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} })
27242024-03-21T18:07:54.037ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
27252024-03-21T18:07:54.037ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
27262024-03-21T18:07:54.037ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
27272024-03-21T18:07:54.037ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
27282024-03-21T18:07:54.037ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
27292024-03-21T18:07:54.037ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
27302024-03-21T18:07:54.037ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
27312024-03-21T18:07:54.037ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27322024-03-21T18:07:54.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27332024-03-21T18:07:54.041ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
27342024-03-21T18:07:54.041ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cbccaebb-fd18-4948-b83b-613dc13392c9", "content-length": "2", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} })
27352024-03-21T18:07:54.041ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
27362024-03-21T18:07:54.041ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "84fab439-9e42-4c21-ae88-788a795aae74", "content-length": "2", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} })
27372024-03-21T18:07:54.042ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
27382024-03-21T18:07:54.042ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27392024-03-21T18:07:54.090ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6aa453a9-5236-4f21-b7c6-6f2df88e7813", "date": "Thu, 21 Mar 2024 18:07:53 GMT"} })
27402024-03-21T18:07:54.090ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
27412024-03-21T18:07:54.090ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27422024-03-21T18:07:54.090ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9a1c2c70-1ecc-4b16-985f-b445bd06d4ae", "content-length": "443", "date": "Thu, 21 Mar 2024 18:07:54 GMT"} })
27432024-03-21T18:07:54.090ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27442024-03-21T18:07:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27452024-03-21T18:07:54.093ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27462024-03-21T18:07:54.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27472024-03-21T18:07:54.099ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 620.102364ms iteration = 10
27482024-03-21T18:07:54.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 739 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = b6f45a05-0a23-4e2a-9fb1-1f3168585c8b response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
27492024-03-21T18:08:03.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
27502024-03-21T18:08:03.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27512024-03-21T18:08:03.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27522024-03-21T18:08:03.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27532024-03-21T18:08:03.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27542024-03-21T18:08:03.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27552024-03-21T18:08:03.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27562024-03-21T18:08:03.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27572024-03-21T18:08:03.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27582024-03-21T18:08:03.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27592024-03-21T18:08:03.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
27602024-03-21T18:08:03.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27612024-03-21T18:08:03.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27622024-03-21T18:08:03.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
27632024-03-21T18:08:03.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.004331ms iteration = 27
27642024-03-21T18:08:04.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 230 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 82033853-5e19-4b39-bc82-fd5b2e86dc34 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
27652024-03-21T18:08:13.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
27662024-03-21T18:08:13.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27672024-03-21T18:08:13.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27682024-03-21T18:08:13.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27692024-03-21T18:08:13.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27702024-03-21T18:08:13.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27712024-03-21T18:08:13.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27722024-03-21T18:08:13.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27732024-03-21T18:08:13.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27742024-03-21T18:08:13.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27752024-03-21T18:08:13.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
27762024-03-21T18:08:13.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27772024-03-21T18:08:13.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27782024-03-21T18:08:13.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
27792024-03-21T18:08:13.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.725496ms iteration = 28
27802024-03-21T18:08:14.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 169 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 6a239c3a-6ca4-4291-8dd1-089db2802e0c response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
27812024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
27822024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
27832024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27842024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
27852024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
27862024-03-21T18:08:23.454ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
27872024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
27882024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27892024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
27902024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27912024-03-21T18:08:23.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27922024-03-21T18:08:23.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27932024-03-21T18:08:23.455ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 181362133 } })
27942024-03-21T18:08:23.456ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
27952024-03-21T18:08:23.456ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.187059ms iteration = 11
27962024-03-21T18:08:23.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
27972024-03-21T18:08:23.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27982024-03-21T18:08:23.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27992024-03-21T18:08:23.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28002024-03-21T18:08:23.459ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
28012024-03-21T18:08:23.459ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
28022024-03-21T18:08:23.459ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
28032024-03-21T18:08:23.459ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 12.01\u{b5}s iteration = 11
28042024-03-21T18:08:23.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28052024-03-21T18:08:23.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28062024-03-21T18:08:23.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28072024-03-21T18:08:23.462ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 188195596 } })
28082024-03-21T18:08:23.463ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
28092024-03-21T18:08:23.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
28102024-03-21T18:08:23.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28112024-03-21T18:08:23.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28122024-03-21T18:08:23.469ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
28132024-03-21T18:08:23.469ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.098501ms iteration = 29
28142024-03-21T18:08:23.479ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
28152024-03-21T18:08:23.479ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
28162024-03-21T18:08:23.479ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28172024-03-21T18:08:23.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28182024-03-21T18:08:23.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28192024-03-21T18:08:23.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28202024-03-21T18:08:23.481ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28212024-03-21T18:08:23.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28222024-03-21T18:08:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28232024-03-21T18:08:23.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28242024-03-21T18:08:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28252024-03-21T18:08:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
28262024-03-21T18:08:23.484ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28272024-03-21T18:08:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28282024-03-21T18:08:23.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28292024-03-21T18:08:23.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28302024-03-21T18:08:23.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 212195106 } })
28312024-03-21T18:08:23.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
28322024-03-21T18:08:23.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28332024-03-21T18:08:23.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28342024-03-21T18:08:23.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28352024-03-21T18:08:23.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28362024-03-21T18:08:23.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28372024-03-21T18:08:23.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
28382024-03-21T18:08:23.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28392024-03-21T18:08:23.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28402024-03-21T18:08:23.506ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.270391ms iteration = 11
28412024-03-21T18:08:23.538ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
28422024-03-21T18:08:23.538ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28432024-03-21T18:08:23.538ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d663c5c7-eb1f-4dab-a5d2-0eeb22d81ad2", "content-length": "26", "date": "Thu, 21 Mar 2024 18:08:23 GMT"} })
28442024-03-21T18:08:23.538ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28452024-03-21T18:08:23.540ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28462024-03-21T18:08:23.625ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
28472024-03-21T18:08:23.625ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28482024-03-21T18:08:23.626ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7161af5f-db1a-4cf8-ae7d-e9b6e9bab480", "content-length": "26", "date": "Thu, 21 Mar 2024 18:08:23 GMT"} })
28492024-03-21T18:08:23.626ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28502024-03-21T18:08:23.627ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28512024-03-21T18:08:23.639ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
28522024-03-21T18:08:23.639ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "15be29eb-7451-4bf4-bdfa-54e445d47bde", "content-length": "1", "date": "Thu, 21 Mar 2024 18:08:22 GMT"} })
28532024-03-21T18:08:23.641ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28542024-03-21T18:08:23.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28552024-03-21T18:08:23.642ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 188.089881ms iteration = 11
28562024-03-21T18:08:23.650ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
28572024-03-21T18:08:23.650ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28582024-03-21T18:08:23.650ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d7003553-27fe-4247-8a8a-6a7f24f31659", "content-length": "26", "date": "Thu, 21 Mar 2024 18:08:23 GMT"} })
28592024-03-21T18:08:23.650ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28602024-03-21T18:08:23.652ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
28612024-03-21T18:08:23.695ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 241.370896ms iteration = 11
28622024-03-21T18:08:23.762ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28632024-03-21T18:08:23.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28642024-03-21T18:08:23.764ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
28652024-03-21T18:08:23.764ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28662024-03-21T18:08:23.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28672024-03-21T18:08:23.771ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
28682024-03-21T18:08:23.771ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
28692024-03-21T18:08:23.771ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "adac630f-b79e-43be-bc17-78555628791b", "content-length": "121", "date": "Thu, 21 Mar 2024 18:08:22 GMT"} })
28702024-03-21T18:08:23.771ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
28712024-03-21T18:08:23.771ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
28722024-03-21T18:08:23.772ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
28732024-03-21T18:08:23.772ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
28742024-03-21T18:08:23.772ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
28752024-03-21T18:08:23.772ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
28762024-03-21T18:08:23.772ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
28772024-03-21T18:08:23.772ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28782024-03-21T18:08:23.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28792024-03-21T18:08:23.773ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
28802024-03-21T18:08:23.774ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "40da8101-ccf6-457b-962b-d7c81dbf3fec", "content-length": "2", "date": "Thu, 21 Mar 2024 18:08:22 GMT"} })
28812024-03-21T18:08:23.774ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
28822024-03-21T18:08:23.774ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6f790021-dbb0-4913-a35a-a3344e7d032a", "content-length": "2", "date": "Thu, 21 Mar 2024 18:08:22 GMT"} })
28832024-03-21T18:08:23.774ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
28842024-03-21T18:08:23.774ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
28852024-03-21T18:08:23.822ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "88edd66d-4cbf-484d-b2ef-a5cb5da11573", "date": "Thu, 21 Mar 2024 18:08:23 GMT"} })
28862024-03-21T18:08:23.822ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
28872024-03-21T18:08:23.822ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
28882024-03-21T18:08:23.822ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "83907670-19e4-4763-b58a-2b97bbd01943", "content-length": "443", "date": "Thu, 21 Mar 2024 18:08:23 GMT"} })
28892024-03-21T18:08:23.822ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28902024-03-21T18:08:23.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28912024-03-21T18:08:23.825ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28922024-03-21T18:08:23.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28932024-03-21T18:08:23.830ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 350.38658ms iteration = 11
28942024-03-21T18:08:24.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 172 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = b345a582-78ba-4f60-a25f-ddfa2602184a response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
28952024-03-21T18:08:33.453ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
28962024-03-21T18:08:33.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28972024-03-21T18:08:33.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28982024-03-21T18:08:33.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28992024-03-21T18:08:33.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29002024-03-21T18:08:33.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29012024-03-21T18:08:33.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29022024-03-21T18:08:33.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29032024-03-21T18:08:33.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29042024-03-21T18:08:33.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29052024-03-21T18:08:33.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
29062024-03-21T18:08:33.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29072024-03-21T18:08:33.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29082024-03-21T18:08:33.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
29092024-03-21T18:08:33.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.174272ms iteration = 30
29102024-03-21T18:08:34.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 261 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = ddfd7af8-ee42-40c5-9afe-38961d73ccfa response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
29112024-03-21T18:08:43.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
29122024-03-21T18:08:43.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29132024-03-21T18:08:43.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29142024-03-21T18:08:43.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29152024-03-21T18:08:43.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29162024-03-21T18:08:43.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29172024-03-21T18:08:43.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29182024-03-21T18:08:43.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29192024-03-21T18:08:43.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29202024-03-21T18:08:43.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29212024-03-21T18:08:43.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
29222024-03-21T18:08:43.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29232024-03-21T18:08:43.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29242024-03-21T18:08:43.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
29252024-03-21T18:08:43.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.026943ms iteration = 31
29262024-03-21T18:08:44.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 717 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 2260095d-e26d-4eae-8666-32902fd107ad response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
29272024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
29282024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29292024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
29302024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29312024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
29322024-03-21T18:08:53.454ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
29332024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
29342024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29352024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
29362024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
29372024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
29382024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
29392024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
29402024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
29412024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
29422024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
29432024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29442024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
29452024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
29462024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29472024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29482024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29492024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29502024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29512024-03-21T18:08:53.454ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 179903957 } })
29522024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29532024-03-21T18:08:53.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
29542024-03-21T18:08:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29552024-03-21T18:08:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29562024-03-21T18:08:53.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29572024-03-21T18:08:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29582024-03-21T18:08:53.456ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 181178900 } })
29592024-03-21T18:08:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29602024-03-21T18:08:53.457ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29612024-03-21T18:08:53.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29622024-03-21T18:08:53.457ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
29632024-03-21T18:08:53.457ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.154536ms iteration = 7
29642024-03-21T18:08:53.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29652024-03-21T18:08:53.457ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
29662024-03-21T18:08:53.457ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.436111ms iteration = 12
29672024-03-21T18:08:53.458ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
29682024-03-21T18:08:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.173675ms iteration = 7
29692024-03-21T18:08:53.459ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
29702024-03-21T18:08:53.459ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
29712024-03-21T18:08:53.459ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
29722024-03-21T18:08:53.459ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.43\u{b5}s iteration = 12
29732024-03-21T18:08:53.460ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 185283254 } })
29742024-03-21T18:08:53.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29752024-03-21T18:08:53.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29762024-03-21T18:08:53.460ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 185455137 } })
29772024-03-21T18:08:53.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29782024-03-21T18:08:53.461ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
29792024-03-21T18:08:53.461ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29802024-03-21T18:08:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29812024-03-21T18:08:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29822024-03-21T18:08:53.462ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
29832024-03-21T18:08:53.462ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
29842024-03-21T18:08:53.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29852024-03-21T18:08:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29862024-03-21T18:08:53.462ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
29872024-03-21T18:08:53.462ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
29882024-03-21T18:08:53.462ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
29892024-03-21T18:08:53.462ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29902024-03-21T18:08:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29912024-03-21T18:08:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29922024-03-21T18:08:53.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29932024-03-21T18:08:53.463ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29942024-03-21T18:08:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29952024-03-21T18:08:53.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29962024-03-21T18:08:53.464ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29972024-03-21T18:08:53.466ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
29982024-03-21T18:08:53.466ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
29992024-03-21T18:08:53.466ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
30002024-03-21T18:08:53.466ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30012024-03-21T18:08:53.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30022024-03-21T18:08:53.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30032024-03-21T18:08:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
30042024-03-21T18:08:53.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30052024-03-21T18:08:53.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
30062024-03-21T18:08:53.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30072024-03-21T18:08:53.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30082024-03-21T18:08:53.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30092024-03-21T18:08:53.469ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
30102024-03-21T18:08:53.469ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
30112024-03-21T18:08:53.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30122024-03-21T18:08:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30132024-03-21T18:08:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30142024-03-21T18:08:53.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30152024-03-21T18:08:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
30162024-03-21T18:08:53.470ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30172024-03-21T18:08:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30182024-03-21T18:08:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30192024-03-21T18:08:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30202024-03-21T18:08:53.472ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
30212024-03-21T18:08:53.472ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.005592ms iteration = 32
30222024-03-21T18:08:53.472ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
30232024-03-21T18:08:53.472ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 0 version = 2
30242024-03-21T18:08:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30252024-03-21T18:08:53.472ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30262024-03-21T18:08:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30272024-03-21T18:08:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30282024-03-21T18:08:53.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30292024-03-21T18:08:53.473ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30302024-03-21T18:08:53.473ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
30312024-03-21T18:08:53.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30322024-03-21T18:08:53.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30332024-03-21T18:08:53.476ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
30342024-03-21T18:08:53.476ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
30352024-03-21T18:08:53.476ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-21 18:04:07.863813 UTC dns_group = internal generation = 1
30362024-03-21T18:08:53.476ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 21.935323ms iteration = 7
30372024-03-21T18:08:53.477ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30382024-03-21T18:08:53.477ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30392024-03-21T18:08:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30402024-03-21T18:08:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30412024-03-21T18:08:53.477ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30422024-03-21T18:08:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30432024-03-21T18:08:53.478ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 1 version = 2
30442024-03-21T18:08:53.478ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
30452024-03-21T18:08:53.478ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-21 18:04:08.259984 UTC dns_group = external generation = 2
30462024-03-21T18:08:53.478ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.295246ms iteration = 7
30472024-03-21T18:08:53.480ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
30482024-03-21T18:08:53.480ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
30492024-03-21T18:08:53.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30502024-03-21T18:08:53.480ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
30512024-03-21T18:08:53.480ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
30522024-03-21T18:08:53.480ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.77\u{b5}s iteration = 8
30532024-03-21T18:08:53.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30542024-03-21T18:08:53.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30552024-03-21T18:08:53.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30562024-03-21T18:08:53.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30572024-03-21T18:08:53.482ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30582024-03-21T18:08:53.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
30592024-03-21T18:08:53.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30602024-03-21T18:08:53.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30612024-03-21T18:08:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30622024-03-21T18:08:53.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30632024-03-21T18:08:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30642024-03-21T18:08:53.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 211180318 } })
30652024-03-21T18:08:53.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30662024-03-21T18:08:53.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30672024-03-21T18:08:53.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
30682024-03-21T18:08:53.489ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-21T18:03:53.561270Z, time_modified: 2024-03-21T18:03:53.561270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }) }
30692024-03-21T18:08:53.489ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 35.465466ms iteration = 7
30702024-03-21T18:08:53.498ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30712024-03-21T18:08:53.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30722024-03-21T18:08:53.500ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30732024-03-21T18:08:53.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30742024-03-21T18:08:53.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30752024-03-21T18:08:53.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
30762024-03-21T18:08:53.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30772024-03-21T18:08:53.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30782024-03-21T18:08:53.504ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 24.00264ms iteration = 12
30792024-03-21T18:08:53.603ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
30802024-03-21T18:08:53.603ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30812024-03-21T18:08:53.604ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f22cba63-1f33-4927-81dd-f1ffcc925e80", "content-length": "26", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} })
30822024-03-21T18:08:53.604ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30832024-03-21T18:08:53.606ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30842024-03-21T18:08:53.616ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
30852024-03-21T18:08:53.616ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30862024-03-21T18:08:53.617ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7f6e1b58-b977-4bc3-87df-d208a2bb8f2e", "content-length": "26", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} })
30872024-03-21T18:08:53.617ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30882024-03-21T18:08:53.619ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30892024-03-21T18:08:53.644ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
30902024-03-21T18:08:53.644ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30912024-03-21T18:08:53.644ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bc8fc3a7-981e-4fa9-b1b2-5c5bb1c7888f", "content-length": "26", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} })
30922024-03-21T18:08:53.644ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30932024-03-21T18:08:53.646ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
30942024-03-21T18:08:53.725ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
30952024-03-21T18:08:53.740ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
30962024-03-21T18:08:53.740ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0ab7d3e5-64f2-4a6f-b0f0-9cf21b1d286a", "content-length": "1", "date": "Thu, 21 Mar 2024 18:08:52 GMT"} })
30972024-03-21T18:08:53.741ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30982024-03-21T18:08:53.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30992024-03-21T18:08:53.742ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 288.46883ms iteration = 12
31002024-03-21T18:08:53.743ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
31012024-03-21T18:08:53.891ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 437.291038ms iteration = 12
31022024-03-21T18:08:53.961ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
31032024-03-21T18:08:53.961ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6bc89daa-19ab-45bb-b928-69ab1e3cc4a5", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31042024-03-21T18:08:53.961ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31052024-03-21T18:08:53.963ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b4ba48f4-16ab-41cc-be5d-a482e87800f8", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31062024-03-21T18:08:53.963ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31072024-03-21T18:08:53.964ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31082024-03-21T18:08:53.964ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 510.51072ms iteration = 9
31092024-03-21T18:08:53.969ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
31102024-03-21T18:08:54.033ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
31112024-03-21T18:08:54.034ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2e908016-a906-4ce5-a923-5245fb6d0e1e", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
31122024-03-21T18:08:54.034ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
31132024-03-21T18:08:54.036ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c50168da-c624-420e-b0f1-3690145d0261", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
31142024-03-21T18:08:54.036ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
31152024-03-21T18:08:54.038ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f19fccf9-55ed-47b6-b4c4-609a149daeec", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
31162024-03-21T18:08:54.038ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
31172024-03-21T18:08:54.040ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
31182024-03-21T18:08:54.040ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 586.072854ms iteration = 9
31192024-03-21T18:08:54.042ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31202024-03-21T18:08:54.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31212024-03-21T18:08:54.044ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
31222024-03-21T18:08:54.044ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31232024-03-21T18:08:54.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31242024-03-21T18:08:54.053ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
31252024-03-21T18:08:54.053ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
31262024-03-21T18:08:54.053ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bb16018e-bfdc-4c1e-a2f8-fc3762efb3cf", "content-length": "121", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} })
31272024-03-21T18:08:54.053ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
31282024-03-21T18:08:54.053ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
31292024-03-21T18:08:54.054ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
31302024-03-21T18:08:54.054ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
31312024-03-21T18:08:54.054ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
31322024-03-21T18:08:54.054ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
31332024-03-21T18:08:54.054ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
31342024-03-21T18:08:54.054ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31352024-03-21T18:08:54.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31362024-03-21T18:08:54.056ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
31372024-03-21T18:08:54.056ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "118f87a4-fef3-4318-9104-028c71c5317f", "content-length": "2", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} })
31382024-03-21T18:08:54.056ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
31392024-03-21T18:08:54.056ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2f4a8a42-7d29-4288-ba9a-5c3902419d65", "content-length": "2", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} })
31402024-03-21T18:08:54.056ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
31412024-03-21T18:08:54.056ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
31422024-03-21T18:08:54.107ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "37ba0edf-9af9-40a9-beb2-1026905050ac", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} })
31432024-03-21T18:08:54.107ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
31442024-03-21T18:08:54.107ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
31452024-03-21T18:08:54.107ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "25dab681-53d7-41bd-a911-d38bf5eb31bd", "content-length": "443", "date": "Thu, 21 Mar 2024 18:08:53 GMT"} })
31462024-03-21T18:08:54.107ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31472024-03-21T18:08:54.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31482024-03-21T18:08:54.110ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31492024-03-21T18:08:54.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31502024-03-21T18:08:54.116ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 636.008519ms iteration = 12
31512024-03-21T18:08:54.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 733 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 0a7ff7b5-524e-4b8e-bc91-79923d2cabe4 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
31522024-03-21T18:09:03.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
31532024-03-21T18:09:03.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31542024-03-21T18:09:03.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31552024-03-21T18:09:03.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31562024-03-21T18:09:03.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31572024-03-21T18:09:03.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31582024-03-21T18:09:03.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31592024-03-21T18:09:03.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31602024-03-21T18:09:03.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31612024-03-21T18:09:03.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31622024-03-21T18:09:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
31632024-03-21T18:09:03.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31642024-03-21T18:09:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31652024-03-21T18:09:03.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
31662024-03-21T18:09:03.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.652814ms iteration = 33
31672024-03-21T18:09:04.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 169 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 6a2ee0fe-cbf3-45ec-ba95-850f83e4fbcd response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
31682024-03-21T18:09:13.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
31692024-03-21T18:09:13.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31702024-03-21T18:09:13.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31712024-03-21T18:09:13.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31722024-03-21T18:09:13.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31732024-03-21T18:09:13.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31742024-03-21T18:09:13.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31752024-03-21T18:09:13.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31762024-03-21T18:09:13.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31772024-03-21T18:09:13.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31782024-03-21T18:09:13.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
31792024-03-21T18:09:13.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31802024-03-21T18:09:13.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31812024-03-21T18:09:13.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
31822024-03-21T18:09:13.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.570713ms iteration = 34
31832024-03-21T18:09:14.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 680 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = f5cecf82-6176-49d2-8ad1-d3d0ed311e80 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
31842024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
31852024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
31862024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31872024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
31882024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31892024-03-21T18:09:23.454ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
31902024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
31912024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
31922024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31932024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31942024-03-21T18:09:23.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
31952024-03-21T18:09:23.456ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 180835163 } })
31962024-03-21T18:09:23.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31972024-03-21T18:09:23.456ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
31982024-03-21T18:09:23.456ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.22432ms iteration = 13
31992024-03-21T18:09:23.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
32002024-03-21T18:09:23.457ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 182227279 } })
32012024-03-21T18:09:23.457ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
32022024-03-21T18:09:23.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32032024-03-21T18:09:23.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32042024-03-21T18:09:23.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32052024-03-21T18:09:23.459ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
32062024-03-21T18:09:23.459ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
32072024-03-21T18:09:23.459ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
32082024-03-21T18:09:23.459ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.071\u{b5}s iteration = 13
32092024-03-21T18:09:23.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32102024-03-21T18:09:23.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32112024-03-21T18:09:23.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32122024-03-21T18:09:23.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
32132024-03-21T18:09:23.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32142024-03-21T18:09:23.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32152024-03-21T18:09:23.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
32162024-03-21T18:09:23.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.805755ms iteration = 35
32172024-03-21T18:09:23.480ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
32182024-03-21T18:09:23.480ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
32192024-03-21T18:09:23.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32202024-03-21T18:09:23.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32212024-03-21T18:09:23.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32222024-03-21T18:09:23.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32232024-03-21T18:09:23.482ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32242024-03-21T18:09:23.482ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32252024-03-21T18:09:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
32262024-03-21T18:09:23.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32272024-03-21T18:09:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32282024-03-21T18:09:23.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
32292024-03-21T18:09:23.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32302024-03-21T18:09:23.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32312024-03-21T18:09:23.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32322024-03-21T18:09:23.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32332024-03-21T18:09:23.488ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 212924209 } })
32342024-03-21T18:09:23.488ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
32352024-03-21T18:09:23.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32362024-03-21T18:09:23.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32372024-03-21T18:09:23.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32382024-03-21T18:09:23.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32392024-03-21T18:09:23.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32402024-03-21T18:09:23.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
32412024-03-21T18:09:23.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32422024-03-21T18:09:23.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32432024-03-21T18:09:23.506ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.438856ms iteration = 13
32442024-03-21T18:09:23.538ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
32452024-03-21T18:09:23.538ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32462024-03-21T18:09:23.538ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d164ce95-e0ee-40a8-a5c2-5c0019a5fa75", "content-length": "26", "date": "Thu, 21 Mar 2024 18:09:23 GMT"} })
32472024-03-21T18:09:23.539ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32482024-03-21T18:09:23.540ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32492024-03-21T18:09:23.614ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
32502024-03-21T18:09:23.614ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32512024-03-21T18:09:23.614ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9ce8c8b3-1994-47a7-bc79-1748fcf7753c", "content-length": "26", "date": "Thu, 21 Mar 2024 18:09:23 GMT"} })
32522024-03-21T18:09:23.614ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32532024-03-21T18:09:23.616ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32542024-03-21T18:09:23.634ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
32552024-03-21T18:09:23.634ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e52512f8-53b7-4337-8ca8-29298f090664", "content-length": "1", "date": "Thu, 21 Mar 2024 18:09:22 GMT"} })
32562024-03-21T18:09:23.635ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32572024-03-21T18:09:23.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32582024-03-21T18:09:23.636ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 182.187595ms iteration = 13
32592024-03-21T18:09:23.639ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
32602024-03-21T18:09:23.639ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32612024-03-21T18:09:23.639ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8ddc0bc6-2842-43a1-b322-c9d1306488bc", "content-length": "26", "date": "Thu, 21 Mar 2024 18:09:23 GMT"} })
32622024-03-21T18:09:23.639ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32632024-03-21T18:09:23.640ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
32642024-03-21T18:09:23.681ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 227.157441ms iteration = 13
32652024-03-21T18:09:23.750ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32662024-03-21T18:09:23.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32672024-03-21T18:09:23.752ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
32682024-03-21T18:09:23.752ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32692024-03-21T18:09:23.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32702024-03-21T18:09:23.758ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
32712024-03-21T18:09:23.758ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
32722024-03-21T18:09:23.759ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "07da6c62-a8a1-4ca6-af00-e010ec5e0401", "content-length": "121", "date": "Thu, 21 Mar 2024 18:09:22 GMT"} })
32732024-03-21T18:09:23.759ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
32742024-03-21T18:09:23.759ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
32752024-03-21T18:09:23.759ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
32762024-03-21T18:09:23.759ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
32772024-03-21T18:09:23.759ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
32782024-03-21T18:09:23.759ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
32792024-03-21T18:09:23.759ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
32802024-03-21T18:09:23.759ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32812024-03-21T18:09:23.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32822024-03-21T18:09:23.761ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
32832024-03-21T18:09:23.761ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "de1b508e-d18f-4016-a88f-109ab1767852", "content-length": "2", "date": "Thu, 21 Mar 2024 18:09:22 GMT"} })
32842024-03-21T18:09:23.761ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
32852024-03-21T18:09:23.761ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7a86c5b7-7986-48ff-8e0c-7f325fb51c38", "content-length": "2", "date": "Thu, 21 Mar 2024 18:09:22 GMT"} })
32862024-03-21T18:09:23.761ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
32872024-03-21T18:09:23.761ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
32882024-03-21T18:09:23.807ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b249d307-707c-4391-8406-cffa664d028a", "date": "Thu, 21 Mar 2024 18:09:23 GMT"} })
32892024-03-21T18:09:23.807ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
32902024-03-21T18:09:23.807ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
32912024-03-21T18:09:23.810ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9c501f7a-a6b0-4392-a4a6-f5e852c04d52", "content-length": "443", "date": "Thu, 21 Mar 2024 18:09:23 GMT"} })
32922024-03-21T18:09:23.810ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32932024-03-21T18:09:23.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32942024-03-21T18:09:23.814ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32952024-03-21T18:09:23.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32962024-03-21T18:09:23.820ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 339.988513ms iteration = 13
32972024-03-21T18:09:24.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 261 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 084df41f-4f1e-4cea-b952-0e9abd42bd05 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
32982024-03-21T18:09:33.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
32992024-03-21T18:09:33.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33002024-03-21T18:09:33.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33012024-03-21T18:09:33.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33022024-03-21T18:09:33.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33032024-03-21T18:09:33.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33042024-03-21T18:09:33.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33052024-03-21T18:09:33.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33062024-03-21T18:09:33.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33072024-03-21T18:09:33.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33082024-03-21T18:09:33.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
33092024-03-21T18:09:33.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33102024-03-21T18:09:33.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33112024-03-21T18:09:33.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
33122024-03-21T18:09:33.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 12.957322ms iteration = 36
33132024-03-21T18:09:34.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 150 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 6ac724b3-45e2-4fbd-9123-1cb85d97b56f response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
33142024-03-21T18:09:43.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
33152024-03-21T18:09:43.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33162024-03-21T18:09:43.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33172024-03-21T18:09:43.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33182024-03-21T18:09:43.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33192024-03-21T18:09:43.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33202024-03-21T18:09:43.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33212024-03-21T18:09:43.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33222024-03-21T18:09:43.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33232024-03-21T18:09:43.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33242024-03-21T18:09:43.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
33252024-03-21T18:09:43.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33262024-03-21T18:09:43.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33272024-03-21T18:09:43.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
33282024-03-21T18:09:43.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.483522ms iteration = 37
33292024-03-21T18:09:44.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 1111 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 0d285da2-c81b-4934-9c62-8c80c81037ae response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
33302024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
33312024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
33322024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33332024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
33342024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33352024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
33362024-03-21T18:09:53.454ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
33372024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
33382024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33392024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
33402024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
33412024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
33422024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33432024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
33442024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
33452024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
33462024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
33472024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
33482024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
33492024-03-21T18:09:53.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33502024-03-21T18:09:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33512024-03-21T18:09:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33522024-03-21T18:09:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33532024-03-21T18:09:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
33542024-03-21T18:09:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33552024-03-21T18:09:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33562024-03-21T18:09:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33572024-03-21T18:09:53.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33582024-03-21T18:09:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33592024-03-21T18:09:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33602024-03-21T18:09:53.456ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 180985346 } })
33612024-03-21T18:09:53.456ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 181068558 } })
33622024-03-21T18:09:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33632024-03-21T18:09:53.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33642024-03-21T18:09:53.457ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
33652024-03-21T18:09:53.457ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.951953ms iteration = 14
33662024-03-21T18:09:53.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33672024-03-21T18:09:53.458ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
33682024-03-21T18:09:53.458ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33692024-03-21T18:09:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.599975ms iteration = 8
33702024-03-21T18:09:53.459ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
33712024-03-21T18:09:53.459ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
33722024-03-21T18:09:53.459ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
33732024-03-21T18:09:53.459ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.71\u{b5}s iteration = 14
33742024-03-21T18:09:53.460ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
33752024-03-21T18:09:53.460ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.859965ms iteration = 8
33762024-03-21T18:09:53.460ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 185234993 } })
33772024-03-21T18:09:53.460ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 185231853 } })
33782024-03-21T18:09:53.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33792024-03-21T18:09:53.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33802024-03-21T18:09:53.461ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
33812024-03-21T18:09:53.461ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33822024-03-21T18:09:53.461ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
33832024-03-21T18:09:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33842024-03-21T18:09:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33852024-03-21T18:09:53.462ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
33862024-03-21T18:09:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
33872024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
33882024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33892024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33902024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
33912024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
33922024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
33932024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33942024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33952024-03-21T18:09:53.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33962024-03-21T18:09:53.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33972024-03-21T18:09:53.464ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
33982024-03-21T18:09:53.464ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
33992024-03-21T18:09:53.464ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
34002024-03-21T18:09:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34012024-03-21T18:09:53.464ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34022024-03-21T18:09:53.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34032024-03-21T18:09:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34042024-03-21T18:09:53.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34052024-03-21T18:09:53.465ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34062024-03-21T18:09:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
34072024-03-21T18:09:53.469ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34082024-03-21T18:09:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34092024-03-21T18:09:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
34102024-03-21T18:09:53.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34112024-03-21T18:09:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
34122024-03-21T18:09:53.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34132024-03-21T18:09:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34142024-03-21T18:09:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34152024-03-21T18:09:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
34162024-03-21T18:09:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34172024-03-21T18:09:53.471ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34182024-03-21T18:09:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34192024-03-21T18:09:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34202024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34212024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
34222024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
34232024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34242024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
34252024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 0 version = 2
34262024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34272024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34282024-03-21T18:09:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34292024-03-21T18:09:53.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34302024-03-21T18:09:53.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34312024-03-21T18:09:53.474ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
34322024-03-21T18:09:53.474ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.101601ms iteration = 38
34332024-03-21T18:09:53.475ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
34342024-03-21T18:09:53.475ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34352024-03-21T18:09:53.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34362024-03-21T18:09:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
34372024-03-21T18:09:53.477ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34382024-03-21T18:09:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34392024-03-21T18:09:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
34402024-03-21T18:09:53.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34412024-03-21T18:09:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34422024-03-21T18:09:53.479ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 1 version = 2
34432024-03-21T18:09:53.479ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
34442024-03-21T18:09:53.479ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-21 18:04:08.259984 UTC dns_group = external generation = 2
34452024-03-21T18:09:53.479ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.180072ms iteration = 8
34462024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
34472024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
34482024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-21 18:04:07.863813 UTC dns_group = internal generation = 1
34492024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 25.450066ms iteration = 8
34502024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
34512024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
34522024-03-21T18:09:53.480ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
34532024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34542024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 4.6\u{b5}s iteration = 9
34552024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
34562024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34572024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34582024-03-21T18:09:53.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34592024-03-21T18:09:53.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34602024-03-21T18:09:53.481ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34612024-03-21T18:09:53.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
34622024-03-21T18:09:53.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34632024-03-21T18:09:53.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34642024-03-21T18:09:53.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
34652024-03-21T18:09:53.484ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34662024-03-21T18:09:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34672024-03-21T18:09:53.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 210840861 } })
34682024-03-21T18:09:53.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34692024-03-21T18:09:53.486ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-21T18:03:53.561270Z, time_modified: 2024-03-21T18:03:53.561270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }) }
34702024-03-21T18:09:53.486ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 32.084495ms iteration = 8
34712024-03-21T18:09:53.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34722024-03-21T18:09:53.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
34732024-03-21T18:09:53.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34742024-03-21T18:09:53.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34752024-03-21T18:09:53.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34762024-03-21T18:09:53.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34772024-03-21T18:09:53.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34782024-03-21T18:09:53.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
34792024-03-21T18:09:53.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34802024-03-21T18:09:53.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34812024-03-21T18:09:53.507ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.898653ms iteration = 14
34822024-03-21T18:09:53.570ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
34832024-03-21T18:09:53.578ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
34842024-03-21T18:09:53.581ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
34852024-03-21T18:09:53.581ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
34862024-03-21T18:09:53.582ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "130e5553-8d79-4ee0-bb9f-b79cf1d37e1f", "content-length": "26", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} })
34872024-03-21T18:09:53.582ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
34882024-03-21T18:09:53.584ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
34892024-03-21T18:09:53.586ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
34902024-03-21T18:09:53.586ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
34912024-03-21T18:09:53.586ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c5b2fc25-3276-4535-8a01-864b31c462e5", "content-length": "26", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} })
34922024-03-21T18:09:53.586ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
34932024-03-21T18:09:53.588ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
34942024-03-21T18:09:53.641ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
34952024-03-21T18:09:53.642ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
34962024-03-21T18:09:53.642ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9d277049-3985-4358-9c1e-42c12fc4cd9b", "content-length": "26", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} })
34972024-03-21T18:09:53.642ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
34982024-03-21T18:09:53.643ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
34992024-03-21T18:09:53.701ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
35002024-03-21T18:09:53.744ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
35012024-03-21T18:09:53.744ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d6d395fa-b7f8-49e1-8bbc-617ff0356d57", "content-length": "1", "date": "Thu, 21 Mar 2024 18:09:52 GMT"} })
35022024-03-21T18:09:53.746ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
35032024-03-21T18:09:53.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
35042024-03-21T18:09:53.748ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 293.50128ms iteration = 14
35052024-03-21T18:09:53.814ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
35062024-03-21T18:09:53.814ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3b53ffaf-d282-49ef-a900-e8aac9585105", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
35072024-03-21T18:09:53.815ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
35082024-03-21T18:09:53.816ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ac124c88-db44-44a9-aad0-dae5d921e306", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
35092024-03-21T18:09:53.816ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
35102024-03-21T18:09:53.818ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8cb91810-6bed-4757-a3ac-d9cc8c4dda55", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
35112024-03-21T18:09:53.818ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
35122024-03-21T18:09:53.820ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
35132024-03-21T18:09:53.820ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 365.919598ms iteration = 10
35142024-03-21T18:09:53.826ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
35152024-03-21T18:09:53.827ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5af379ec-2f9f-49d2-8fca-244fc062f5d9", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35162024-03-21T18:09:53.827ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35172024-03-21T18:09:53.828ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "05824653-574c-4912-a8dc-e96356b2c602", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35182024-03-21T18:09:53.828ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35192024-03-21T18:09:53.830ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 375.324707ms iteration = 14
35202024-03-21T18:09:53.830ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35212024-03-21T18:09:53.830ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 375.52418ms iteration = 10
35222024-03-21T18:09:53.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35232024-03-21T18:09:53.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35242024-03-21T18:09:53.870ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
35252024-03-21T18:09:53.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35262024-03-21T18:09:53.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35272024-03-21T18:09:53.878ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
35282024-03-21T18:09:53.878ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
35292024-03-21T18:09:53.878ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "546c5416-805a-4c78-835c-cf406f00390a", "content-length": "121", "date": "Thu, 21 Mar 2024 18:09:52 GMT"} })
35302024-03-21T18:09:53.878ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
35312024-03-21T18:09:53.878ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
35322024-03-21T18:09:53.879ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
35332024-03-21T18:09:53.879ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
35342024-03-21T18:09:53.879ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
35352024-03-21T18:09:53.879ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
35362024-03-21T18:09:53.879ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
35372024-03-21T18:09:53.879ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35382024-03-21T18:09:53.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35392024-03-21T18:09:53.881ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
35402024-03-21T18:09:53.881ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8be7361a-6a7e-4673-8b1d-a6049894865d", "content-length": "2", "date": "Thu, 21 Mar 2024 18:09:52 GMT"} })
35412024-03-21T18:09:53.881ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
35422024-03-21T18:09:53.881ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "67cd02cb-308d-4df8-9900-3f886c81a330", "content-length": "52", "date": "Thu, 21 Mar 2024 18:09:52 GMT"} })
35432024-03-21T18:09:53.881ZINFOnexus (BackgroundTasks): deleting loopback addresses addresses = [(Switch0, fd38:3181:a596:60e6::1)] background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
35442024-03-21T18:09:53.881ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = DELETE rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6/fd38:3181:a596:60e6::1
35452024-03-21T18:09:53.881ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6/fd38:3181:a596:60e6::1", query: None, fragment: None }, status: 204, headers: {"x-request-id": "11ff0e34-e37f-4ad1-9785-49dad083b33d", "date": "Thu, 21 Mar 2024 18:09:52 GMT"} })
35462024-03-21T18:09:53.881ZINFOnexus (BackgroundTasks): deleted loopback address address = fd38:3181:a596:60e6::1 background_task = switch_port_config_manager file = clients/dpd-client/src/lib.rs:245 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
35472024-03-21T18:09:53.881ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
35482024-03-21T18:09:53.881ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
35492024-03-21T18:09:53.929ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "47672821-d8ee-4989-ab67-4f1ad96ea9bf", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} })
35502024-03-21T18:09:53.929ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
35512024-03-21T18:09:53.929ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
35522024-03-21T18:09:53.929ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ccfe019d-0a91-4e63-bfc3-7669a1372bc2", "content-length": "443", "date": "Thu, 21 Mar 2024 18:09:53 GMT"} })
35532024-03-21T18:09:53.929ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35542024-03-21T18:09:53.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35552024-03-21T18:09:53.932ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35562024-03-21T18:09:53.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35572024-03-21T18:09:53.939ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 458.491827ms iteration = 14
35582024-03-21T18:09:54.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 238 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 6e9f0b00-7bce-49da-b8f9-30c512e7b272 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
35592024-03-21T18:10:03.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
35602024-03-21T18:10:03.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35612024-03-21T18:10:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35622024-03-21T18:10:03.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35632024-03-21T18:10:03.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35642024-03-21T18:10:03.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35652024-03-21T18:10:03.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35662024-03-21T18:10:03.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35672024-03-21T18:10:03.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35682024-03-21T18:10:03.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35692024-03-21T18:10:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
35702024-03-21T18:10:03.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35712024-03-21T18:10:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35722024-03-21T18:10:03.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
35732024-03-21T18:10:03.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.39985ms iteration = 39
35742024-03-21T18:10:04.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 180 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = fa93c819-0d65-48a5-8eef-d05f1732a8c8 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
35752024-03-21T18:10:13.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
35762024-03-21T18:10:13.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35772024-03-21T18:10:13.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35782024-03-21T18:10:13.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35792024-03-21T18:10:13.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35802024-03-21T18:10:13.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35812024-03-21T18:10:13.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35822024-03-21T18:10:13.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35832024-03-21T18:10:13.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35842024-03-21T18:10:13.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35852024-03-21T18:10:13.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
35862024-03-21T18:10:13.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35872024-03-21T18:10:13.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35882024-03-21T18:10:13.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
35892024-03-21T18:10:13.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.075724ms iteration = 40
35902024-03-21T18:10:14.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 877 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 926dd672-b6e0-4c86-9bc8-54a5e57adfc4 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
35912024-03-21T18:10:17.013ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1:0]:56966
35922024-03-21T18:10:17.060ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = c081ffa7-57b2-4200-b25f-9fd86c55e543
35932024-03-21T18:10:17.066ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = c081ffa7-57b2-4200-b25f-9fd86c55e543 uri = http://[fd00:1122:3344:101::d]:12223/producers
35942024-03-21T18:10:17.066ZDEBGnexus: client response oximeter-collector = c081ffa7-57b2-4200-b25f-9fd86c55e543 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "59205a34-56ff-4d58-80aa-f0538eee00b1", "date": "Thu, 21 Mar 2024 18:10:17 GMT"} })
35952024-03-21T18:10:17.066ZINFOnexus: assigned collector to new producer collector_id = c081ffa7-57b2-4200-b25f-9fd86c55e543 file = nexus/src/app/oximeter.rs:193 producer_id = 6dab969d-aefa-442f-abdc-77734bc13f1a
35962024-03-21T18:10:17.068ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 54268 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:56966 req_id = f7d9d6a5-137c-4f96-b31b-3e478ba49708 response_code = 204 uri = /metrics/producers
35972024-03-21T18:10:23.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
35982024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
35992024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36002024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
36012024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36022024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
36032024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
36042024-03-21T18:10:23.455ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
36052024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36062024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
36072024-03-21T18:10:23.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36082024-03-21T18:10:23.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36092024-03-21T18:10:23.457ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 181264262 } })
36102024-03-21T18:10:23.457ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
36112024-03-21T18:10:23.457ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.312271ms iteration = 15
36122024-03-21T18:10:23.457ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
36132024-03-21T18:10:23.457ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 182143407 } })
36142024-03-21T18:10:23.458ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
36152024-03-21T18:10:23.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36162024-03-21T18:10:23.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36172024-03-21T18:10:23.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36182024-03-21T18:10:23.459ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
36192024-03-21T18:10:23.459ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
36202024-03-21T18:10:23.459ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
36212024-03-21T18:10:23.459ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.581\u{b5}s iteration = 15
36222024-03-21T18:10:23.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36232024-03-21T18:10:23.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36242024-03-21T18:10:23.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36252024-03-21T18:10:23.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
36262024-03-21T18:10:23.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36272024-03-21T18:10:23.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36282024-03-21T18:10:23.471ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
36292024-03-21T18:10:23.471ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.417814ms iteration = 41
36302024-03-21T18:10:23.480ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
36312024-03-21T18:10:23.480ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
36322024-03-21T18:10:23.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36332024-03-21T18:10:23.480ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36342024-03-21T18:10:23.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36352024-03-21T18:10:23.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36362024-03-21T18:10:23.481ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36372024-03-21T18:10:23.482ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36382024-03-21T18:10:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
36392024-03-21T18:10:23.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36402024-03-21T18:10:23.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36412024-03-21T18:10:23.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
36422024-03-21T18:10:23.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36432024-03-21T18:10:23.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36442024-03-21T18:10:23.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36452024-03-21T18:10:23.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 210902223 } })
36462024-03-21T18:10:23.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36472024-03-21T18:10:23.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
36482024-03-21T18:10:23.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36492024-03-21T18:10:23.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36502024-03-21T18:10:23.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36512024-03-21T18:10:23.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36522024-03-21T18:10:23.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36532024-03-21T18:10:23.504ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
36542024-03-21T18:10:23.504ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36552024-03-21T18:10:23.504ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
36562024-03-21T18:10:23.504ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36572024-03-21T18:10:23.504ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "296bd484-e162-41b5-aca6-e56cf593821a", "content-length": "26", "date": "Thu, 21 Mar 2024 18:10:23 GMT"} })
36582024-03-21T18:10:23.504ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36592024-03-21T18:10:23.504ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6fe1c25a-54fc-44c3-b57b-f8260b43180a", "content-length": "26", "date": "Thu, 21 Mar 2024 18:10:23 GMT"} })
36602024-03-21T18:10:23.504ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36612024-03-21T18:10:23.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
36622024-03-21T18:10:23.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36632024-03-21T18:10:23.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36642024-03-21T18:10:23.505ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36652024-03-21T18:10:23.505ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36662024-03-21T18:10:23.506ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.475716ms iteration = 15
36672024-03-21T18:10:23.534ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
36682024-03-21T18:10:23.534ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36692024-03-21T18:10:23.534ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a41b493a-3a79-4ddf-898a-af56e3994967", "content-length": "26", "date": "Thu, 21 Mar 2024 18:10:23 GMT"} })
36702024-03-21T18:10:23.534ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36712024-03-21T18:10:23.535ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
36722024-03-21T18:10:23.553ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 98.552487ms iteration = 15
36732024-03-21T18:10:23.553ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
36742024-03-21T18:10:23.554ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "941ca796-4adc-4ad1-b50e-a33018385bd5", "content-length": "1", "date": "Thu, 21 Mar 2024 18:10:22 GMT"} })
36752024-03-21T18:10:23.554ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36762024-03-21T18:10:23.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36772024-03-21T18:10:23.555ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 100.720835ms iteration = 15
36782024-03-21T18:10:23.630ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36792024-03-21T18:10:23.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36802024-03-21T18:10:23.632ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
36812024-03-21T18:10:23.632ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36822024-03-21T18:10:23.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36832024-03-21T18:10:23.639ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
36842024-03-21T18:10:23.639ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
36852024-03-21T18:10:23.640ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a0a9683b-21d7-4425-9b23-88fd93d3de30", "content-length": "121", "date": "Thu, 21 Mar 2024 18:10:22 GMT"} })
36862024-03-21T18:10:23.640ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
36872024-03-21T18:10:23.640ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
36882024-03-21T18:10:23.640ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
36892024-03-21T18:10:23.640ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
36902024-03-21T18:10:23.640ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
36912024-03-21T18:10:23.640ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
36922024-03-21T18:10:23.640ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
36932024-03-21T18:10:23.640ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36942024-03-21T18:10:23.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36952024-03-21T18:10:23.641ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
36962024-03-21T18:10:23.642ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1ca37f6d-eb13-4b63-91f8-f7dd116b717b", "content-length": "2", "date": "Thu, 21 Mar 2024 18:10:22 GMT"} })
36972024-03-21T18:10:23.642ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
36982024-03-21T18:10:23.642ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cce1bb5e-32df-45ca-a3a2-80565aa31ccd", "content-length": "52", "date": "Thu, 21 Mar 2024 18:10:22 GMT"} })
36992024-03-21T18:10:23.642ZINFOnexus (BackgroundTasks): deleting loopback addresses addresses = [(Switch0, fd38:3181:a596:60e6::1)] background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
37002024-03-21T18:10:23.642ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = DELETE rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6/fd38:3181:a596:60e6::1
37012024-03-21T18:10:23.642ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6/fd38:3181:a596:60e6::1", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2ebc7911-1c5f-4e72-8a00-6368fbd8021f", "date": "Thu, 21 Mar 2024 18:10:22 GMT"} })
37022024-03-21T18:10:23.642ZINFOnexus (BackgroundTasks): deleted loopback address address = fd38:3181:a596:60e6::1 background_task = switch_port_config_manager file = clients/dpd-client/src/lib.rs:245 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
37032024-03-21T18:10:23.642ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
37042024-03-21T18:10:23.642ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
37052024-03-21T18:10:23.687ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e77af9df-a8a5-41fd-a42b-5667d34afdd7", "date": "Thu, 21 Mar 2024 18:10:23 GMT"} })
37062024-03-21T18:10:23.687ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
37072024-03-21T18:10:23.687ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
37082024-03-21T18:10:23.688ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0b56472d-46e6-454f-a366-d92c554e871c", "content-length": "443", "date": "Thu, 21 Mar 2024 18:10:23 GMT"} })
37092024-03-21T18:10:23.688ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37102024-03-21T18:10:23.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37112024-03-21T18:10:23.690ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37122024-03-21T18:10:23.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37132024-03-21T18:10:23.698ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 217.383136ms iteration = 15
37142024-03-21T18:10:24.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 1132 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 3f70f5f1-2f75-42e6-a84b-50f5ec8a4598 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
37152024-03-21T18:10:33.454ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
37162024-03-21T18:10:33.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37172024-03-21T18:10:33.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37182024-03-21T18:10:33.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37192024-03-21T18:10:33.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37202024-03-21T18:10:33.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37212024-03-21T18:10:33.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37222024-03-21T18:10:33.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37232024-03-21T18:10:33.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37242024-03-21T18:10:33.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37252024-03-21T18:10:33.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
37262024-03-21T18:10:33.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37272024-03-21T18:10:33.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37282024-03-21T18:10:33.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
37292024-03-21T18:10:33.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.850368ms iteration = 42
37302024-03-21T18:10:34.127ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 146 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = f1e6d1d0-af30-42ad-a1e0-97bdee1aee89 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
37312024-03-21T18:10:43.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
37322024-03-21T18:10:43.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37332024-03-21T18:10:43.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37342024-03-21T18:10:43.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37352024-03-21T18:10:43.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37362024-03-21T18:10:43.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37372024-03-21T18:10:43.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37382024-03-21T18:10:43.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37392024-03-21T18:10:43.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37402024-03-21T18:10:43.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37412024-03-21T18:10:43.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
37422024-03-21T18:10:43.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37432024-03-21T18:10:43.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37442024-03-21T18:10:43.469ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
37452024-03-21T18:10:43.469ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.900419ms iteration = 43
37462024-03-21T18:10:44.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 180 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 59f6d5e6-aa6d-40e7-a8ef-ed4b1a6c59eb response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
37472024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
37482024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
37492024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37502024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
37512024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37522024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
37532024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
37542024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37552024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
37562024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
37572024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
37582024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37592024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
37602024-03-21T18:10:53.455ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
37612024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
37622024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
37632024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
37642024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
37652024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
37662024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
37672024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37682024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37692024-03-21T18:10:53.455ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 179790655 } })
37702024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37712024-03-21T18:10:53.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
37722024-03-21T18:10:53.456ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 179914747 } })
37732024-03-21T18:10:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
37742024-03-21T18:10:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37752024-03-21T18:10:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37762024-03-21T18:10:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37772024-03-21T18:10:53.457ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37782024-03-21T18:10:53.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37792024-03-21T18:10:53.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37802024-03-21T18:10:53.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37812024-03-21T18:10:53.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37822024-03-21T18:10:53.458ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
37832024-03-21T18:10:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.80616ms iteration = 16
37842024-03-21T18:10:53.458ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
37852024-03-21T18:10:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.241618ms iteration = 9
37862024-03-21T18:10:53.458ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
37872024-03-21T18:10:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.398071ms iteration = 9
37882024-03-21T18:10:53.458ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37892024-03-21T18:10:53.459ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 183485111 } })
37902024-03-21T18:10:53.459ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 183652954 } })
37912024-03-21T18:10:53.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
37922024-03-21T18:10:53.459ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37932024-03-21T18:10:53.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37942024-03-21T18:10:53.460ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
37952024-03-21T18:10:53.460ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
37962024-03-21T18:10:53.460ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
37972024-03-21T18:10:53.460ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.74\u{b5}s iteration = 16
37982024-03-21T18:10:53.460ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
37992024-03-21T18:10:53.460ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
38002024-03-21T18:10:53.461ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38012024-03-21T18:10:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38022024-03-21T18:10:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38032024-03-21T18:10:53.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38042024-03-21T18:10:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38052024-03-21T18:10:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38062024-03-21T18:10:53.463ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38072024-03-21T18:10:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
38082024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38092024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
38102024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
38112024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
38122024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38132024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38142024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
38152024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
38162024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
38172024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38182024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38192024-03-21T18:10:53.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38202024-03-21T18:10:53.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38212024-03-21T18:10:53.465ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38222024-03-21T18:10:53.465ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38232024-03-21T18:10:53.466ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
38242024-03-21T18:10:53.466ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38252024-03-21T18:10:53.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38262024-03-21T18:10:53.468ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38272024-03-21T18:10:53.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38282024-03-21T18:10:53.469ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38292024-03-21T18:10:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38302024-03-21T18:10:53.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38312024-03-21T18:10:53.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38322024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
38332024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
38342024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38352024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
38362024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38372024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38382024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
38392024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38402024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38412024-03-21T18:10:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38422024-03-21T18:10:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38432024-03-21T18:10:53.472ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
38442024-03-21T18:10:53.472ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38452024-03-21T18:10:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38462024-03-21T18:10:53.473ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
38472024-03-21T18:10:53.473ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 0 version = 2
38482024-03-21T18:10:53.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38492024-03-21T18:10:53.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38502024-03-21T18:10:53.474ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38512024-03-21T18:10:53.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38522024-03-21T18:10:53.474ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38532024-03-21T18:10:53.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38542024-03-21T18:10:53.475ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
38552024-03-21T18:10:53.475ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.560439ms iteration = 44
38562024-03-21T18:10:53.476ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
38572024-03-21T18:10:53.476ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
38582024-03-21T18:10:53.476ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-21 18:04:07.863813 UTC dns_group = internal generation = 1
38592024-03-21T18:10:53.476ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 21.423834ms iteration = 9
38602024-03-21T18:10:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
38612024-03-21T18:10:53.477ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38622024-03-21T18:10:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38632024-03-21T18:10:53.478ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 1 version = 2
38642024-03-21T18:10:53.478ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
38652024-03-21T18:10:53.478ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-21 18:04:08.259984 UTC dns_group = external generation = 2
38662024-03-21T18:10:53.478ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.44417ms iteration = 9
38672024-03-21T18:10:53.480ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-21T18:03:53.561270Z, time_modified: 2024-03-21T18:03:53.561270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
38682024-03-21T18:10:53.480ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 25.504027ms iteration = 9
38692024-03-21T18:10:53.481ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
38702024-03-21T18:10:53.481ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
38712024-03-21T18:10:53.482ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
38722024-03-21T18:10:53.482ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38732024-03-21T18:10:53.482ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 5.35\u{b5}s iteration = 10
38742024-03-21T18:10:53.482ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
38752024-03-21T18:10:53.482ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38762024-03-21T18:10:53.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38772024-03-21T18:10:53.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38782024-03-21T18:10:53.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38792024-03-21T18:10:53.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38802024-03-21T18:10:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
38812024-03-21T18:10:53.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38822024-03-21T18:10:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38832024-03-21T18:10:53.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
38842024-03-21T18:10:53.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38852024-03-21T18:10:53.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38862024-03-21T18:10:53.487ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38872024-03-21T18:10:53.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 211687987 } })
38882024-03-21T18:10:53.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38892024-03-21T18:10:53.488ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
38902024-03-21T18:10:53.510ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38912024-03-21T18:10:53.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38922024-03-21T18:10:53.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38932024-03-21T18:10:53.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38942024-03-21T18:10:53.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38952024-03-21T18:10:53.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
38962024-03-21T18:10:53.515ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38972024-03-21T18:10:53.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38982024-03-21T18:10:53.516ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 34.526859ms iteration = 16
38992024-03-21T18:10:53.606ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
39002024-03-21T18:10:53.718ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
39012024-03-21T18:10:53.719ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
39022024-03-21T18:10:53.719ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6d656946-f525-4cb0-b9f7-7e79d8a7e553", "content-length": "26", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} })
39032024-03-21T18:10:53.719ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
39042024-03-21T18:10:53.720ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
39052024-03-21T18:10:53.735ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
39062024-03-21T18:10:53.754ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
39072024-03-21T18:10:53.754ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
39082024-03-21T18:10:53.755ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "12fa6345-4eb7-4637-ad4f-2ef2e6e83e26", "content-length": "26", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} })
39092024-03-21T18:10:53.755ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
39102024-03-21T18:10:53.757ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
39112024-03-21T18:10:53.758ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
39122024-03-21T18:10:53.760ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
39132024-03-21T18:10:53.760ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
39142024-03-21T18:10:53.760ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "87ef7f66-0000-44b2-b047-ad6e70f66352", "content-length": "26", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} })
39152024-03-21T18:10:53.760ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
39162024-03-21T18:10:53.761ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
39172024-03-21T18:10:53.918ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
39182024-03-21T18:10:53.918ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "99f4c7a8-a49d-47dc-9268-3e69253415d8", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39192024-03-21T18:10:53.918ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39202024-03-21T18:10:53.919ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a5ce1fcc-5b10-4245-8fb3-696501590233", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39212024-03-21T18:10:53.919ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39222024-03-21T18:10:53.921ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c37890e2-b03f-40a0-abc6-e93ad1deda0d", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39232024-03-21T18:10:53.921ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39242024-03-21T18:10:53.923ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
39252024-03-21T18:10:53.923ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 468.270062ms iteration = 11
39262024-03-21T18:10:53.951ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 495.991049ms iteration = 16
39272024-03-21T18:10:53.981ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
39282024-03-21T18:10:53.982ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a0991037-9901-4854-9ef2-6fcfae474e78", "content-length": "2", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} })
39292024-03-21T18:10:53.984ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
39302024-03-21T18:10:53.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
39312024-03-21T18:10:53.985ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 530.132341ms iteration = 16
39322024-03-21T18:10:53.998ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
39332024-03-21T18:10:53.998ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f99376e7-2554-4316-973c-1224e7676442", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39342024-03-21T18:10:53.998ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39352024-03-21T18:10:54.000ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fda93817-173f-4332-a73a-247cb1f74677", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39362024-03-21T18:10:54.000ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39372024-03-21T18:10:54.002ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39382024-03-21T18:10:54.002ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 546.785939ms iteration = 11
39392024-03-21T18:10:54.058ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39402024-03-21T18:10:54.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39412024-03-21T18:10:54.060ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
39422024-03-21T18:10:54.060ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39432024-03-21T18:10:54.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39442024-03-21T18:10:54.070ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
39452024-03-21T18:10:54.070ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
39462024-03-21T18:10:54.070ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8203ed82-40b6-4121-b33e-209a0167e122", "content-length": "121", "date": "Thu, 21 Mar 2024 18:10:54 GMT"} })
39472024-03-21T18:10:54.070ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
39482024-03-21T18:10:54.070ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
39492024-03-21T18:10:54.070ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
39502024-03-21T18:10:54.070ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
39512024-03-21T18:10:54.070ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
39522024-03-21T18:10:54.070ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
39532024-03-21T18:10:54.070ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
39542024-03-21T18:10:54.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39552024-03-21T18:10:54.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39562024-03-21T18:10:54.072ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
39572024-03-21T18:10:54.072ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c1eab357-42b6-4194-929c-158007263631", "content-length": "2", "date": "Thu, 21 Mar 2024 18:10:54 GMT"} })
39582024-03-21T18:10:54.072ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
39592024-03-21T18:10:54.072ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1ed78241-c11a-4c23-acb0-70361fa5d1f0", "content-length": "52", "date": "Thu, 21 Mar 2024 18:10:54 GMT"} })
39602024-03-21T18:10:54.072ZINFOnexus (BackgroundTasks): deleting loopback addresses addresses = [(Switch0, fd38:3181:a596:60e6::1)] background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
39612024-03-21T18:10:54.072ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = DELETE rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6/fd38:3181:a596:60e6::1
39622024-03-21T18:10:54.073ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6/fd38:3181:a596:60e6::1", query: None, fragment: None }, status: 204, headers: {"x-request-id": "cb9c549b-1215-4c94-98f0-0c85fdb93839", "date": "Thu, 21 Mar 2024 18:10:54 GMT"} })
39632024-03-21T18:10:54.073ZINFOnexus (BackgroundTasks): deleted loopback address address = fd38:3181:a596:60e6::1 background_task = switch_port_config_manager file = clients/dpd-client/src/lib.rs:245 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
39642024-03-21T18:10:54.073ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
39652024-03-21T18:10:54.073ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
39662024-03-21T18:10:54.119ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "34e10d02-8016-4d30-8a28-a1f62b75fa4f", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} })
39672024-03-21T18:10:54.119ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
39682024-03-21T18:10:54.119ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
39692024-03-21T18:10:54.123ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "75856357-ea02-4c9f-b0f9-de2007d38527", "content-length": "443", "date": "Thu, 21 Mar 2024 18:10:53 GMT"} })
39702024-03-21T18:10:54.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39712024-03-21T18:10:54.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39722024-03-21T18:10:54.126ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39732024-03-21T18:10:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39742024-03-21T18:10:54.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 142 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 05b6d2ec-16df-4eb6-b9a2-259dbdae9329 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
39752024-03-21T18:10:54.133ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 651.997055ms iteration = 16
39762024-03-21T18:11:03.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
39772024-03-21T18:11:03.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39782024-03-21T18:11:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39792024-03-21T18:11:03.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39802024-03-21T18:11:03.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39812024-03-21T18:11:03.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39822024-03-21T18:11:03.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39832024-03-21T18:11:03.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39842024-03-21T18:11:03.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39852024-03-21T18:11:03.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39862024-03-21T18:11:03.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
39872024-03-21T18:11:03.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39882024-03-21T18:11:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39892024-03-21T18:11:03.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
39902024-03-21T18:11:03.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 12.778909ms iteration = 45
39912024-03-21T18:11:04.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 250 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 041defc4-90bd-4afd-a3f3-063b1f61cd7e response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
39922024-03-21T18:11:13.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
39932024-03-21T18:11:13.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39942024-03-21T18:11:13.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39952024-03-21T18:11:13.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39962024-03-21T18:11:13.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39972024-03-21T18:11:13.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39982024-03-21T18:11:13.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39992024-03-21T18:11:13.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40002024-03-21T18:11:13.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40012024-03-21T18:11:13.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40022024-03-21T18:11:13.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
40032024-03-21T18:11:13.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40042024-03-21T18:11:13.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40052024-03-21T18:11:13.469ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
40062024-03-21T18:11:13.469ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.690984ms iteration = 46
40072024-03-21T18:11:14.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 253 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 11cddef7-2c78-48f5-a368-1c1a3714a091 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
40082024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
40092024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
40102024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40112024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
40122024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
40132024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
40142024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
40152024-03-21T18:11:23.455ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
40162024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
40172024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40182024-03-21T18:11:23.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
40192024-03-21T18:11:23.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40202024-03-21T18:11:23.457ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
40212024-03-21T18:11:23.457ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.154338ms iteration = 17
40222024-03-21T18:11:23.457ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 181095548 } })
40232024-03-21T18:11:23.457ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 181145479 } })
40242024-03-21T18:11:23.458ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
40252024-03-21T18:11:23.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40262024-03-21T18:11:23.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40272024-03-21T18:11:23.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40282024-03-21T18:11:23.460ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
40292024-03-21T18:11:23.460ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
40302024-03-21T18:11:23.460ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
40312024-03-21T18:11:23.460ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
40322024-03-21T18:11:23.460ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 10.261\u{b5}s iteration = 17
40332024-03-21T18:11:23.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40342024-03-21T18:11:23.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40352024-03-21T18:11:23.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40362024-03-21T18:11:23.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
40372024-03-21T18:11:23.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40382024-03-21T18:11:23.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40392024-03-21T18:11:23.469ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
40402024-03-21T18:11:23.469ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.582902ms iteration = 47
40412024-03-21T18:11:23.481ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
40422024-03-21T18:11:23.481ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
40432024-03-21T18:11:23.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40442024-03-21T18:11:23.481ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40452024-03-21T18:11:23.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40462024-03-21T18:11:23.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40472024-03-21T18:11:23.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40482024-03-21T18:11:23.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40492024-03-21T18:11:23.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
40502024-03-21T18:11:23.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40512024-03-21T18:11:23.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40522024-03-21T18:11:23.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
40532024-03-21T18:11:23.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40542024-03-21T18:11:23.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40552024-03-21T18:11:23.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 211513803 } })
40562024-03-21T18:11:23.488ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40572024-03-21T18:11:23.488ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
40582024-03-21T18:11:23.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40592024-03-21T18:11:23.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40602024-03-21T18:11:23.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40612024-03-21T18:11:23.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40622024-03-21T18:11:23.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40632024-03-21T18:11:23.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40642024-03-21T18:11:23.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
40652024-03-21T18:11:23.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40662024-03-21T18:11:23.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40672024-03-21T18:11:23.505ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 23.742875ms iteration = 17
40682024-03-21T18:11:23.535ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
40692024-03-21T18:11:23.535ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40702024-03-21T18:11:23.535ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "256accd2-7d32-434f-8a37-69abba4a92f5", "content-length": "26", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
40712024-03-21T18:11:23.535ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40722024-03-21T18:11:23.537ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40732024-03-21T18:11:23.598ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
40742024-03-21T18:11:23.598ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40752024-03-21T18:11:23.599ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f61863c6-b2a2-4293-a410-e3d61350cc18", "content-length": "26", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
40762024-03-21T18:11:23.599ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40772024-03-21T18:11:23.601ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40782024-03-21T18:11:23.629ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 173.905117ms iteration = 17
40792024-03-21T18:11:23.632ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
40802024-03-21T18:11:23.632ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40812024-03-21T18:11:23.632ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2c37d52f-200d-470e-8bdb-9cc63eff7e97", "content-length": "26", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
40822024-03-21T18:11:23.632ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40832024-03-21T18:11:23.634ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
40842024-03-21T18:11:23.663ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
40852024-03-21T18:11:23.663ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "043a4792-a42d-4577-b604-8d90d40dd527", "content-length": "2", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
40862024-03-21T18:11:23.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
40872024-03-21T18:11:23.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
40882024-03-21T18:11:23.665ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 210.140507ms iteration = 17
40892024-03-21T18:11:23.742ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40902024-03-21T18:11:23.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40912024-03-21T18:11:23.744ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
40922024-03-21T18:11:23.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40932024-03-21T18:11:23.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40942024-03-21T18:11:23.750ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
40952024-03-21T18:11:23.750ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
40962024-03-21T18:11:23.751ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "720e7686-0699-4787-b943-de2d41089a76", "content-length": "121", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
40972024-03-21T18:11:23.751ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
40982024-03-21T18:11:23.751ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
40992024-03-21T18:11:23.751ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
41002024-03-21T18:11:23.751ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
41012024-03-21T18:11:23.751ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
41022024-03-21T18:11:23.751ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
41032024-03-21T18:11:23.751ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
41042024-03-21T18:11:23.751ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41052024-03-21T18:11:23.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41062024-03-21T18:11:23.753ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
41072024-03-21T18:11:23.753ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1bdef3d7-a70e-4a1c-a039-54bfff8e9a22", "content-length": "2", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
41082024-03-21T18:11:23.753ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
41092024-03-21T18:11:23.753ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fc68b6d9-7fcc-4a3a-a5db-703218462f6f", "content-length": "2", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
41102024-03-21T18:11:23.753ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
41112024-03-21T18:11:23.753ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
41122024-03-21T18:11:23.801ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9cf6075f-4090-4d58-a46e-96de50dc3f2b", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
41132024-03-21T18:11:23.801ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
41142024-03-21T18:11:23.801ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
41152024-03-21T18:11:23.801ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c02b4cab-23ff-434e-847d-e31aeabec509", "content-length": "443", "date": "Thu, 21 Mar 2024 18:11:23 GMT"} })
41162024-03-21T18:11:23.802ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41172024-03-21T18:11:23.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41182024-03-21T18:11:23.804ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41192024-03-21T18:11:23.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41202024-03-21T18:11:23.809ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 327.495079ms iteration = 17
41212024-03-21T18:11:24.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 173 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 0a6bc82c-9037-4942-912a-9bc2e28cff7d response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
41222024-03-21T18:11:33.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
41232024-03-21T18:11:33.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41242024-03-21T18:11:33.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41252024-03-21T18:11:33.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41262024-03-21T18:11:33.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41272024-03-21T18:11:33.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41282024-03-21T18:11:33.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41292024-03-21T18:11:33.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41302024-03-21T18:11:33.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41312024-03-21T18:11:33.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41322024-03-21T18:11:33.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
41332024-03-21T18:11:33.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41342024-03-21T18:11:33.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41352024-03-21T18:11:33.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
41362024-03-21T18:11:33.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.234368ms iteration = 48
41372024-03-21T18:11:34.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 1052 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 29e8ae1e-2e4a-4e74-81f5-5a88d883f1ec response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
41382024-03-21T18:11:43.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
41392024-03-21T18:11:43.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41402024-03-21T18:11:43.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41412024-03-21T18:11:43.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41422024-03-21T18:11:43.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41432024-03-21T18:11:43.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41442024-03-21T18:11:43.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41452024-03-21T18:11:43.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41462024-03-21T18:11:43.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41472024-03-21T18:11:43.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41482024-03-21T18:11:43.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
41492024-03-21T18:11:43.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41502024-03-21T18:11:43.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41512024-03-21T18:11:43.469ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
41522024-03-21T18:11:43.469ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.787077ms iteration = 49
41532024-03-21T18:11:44.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 225 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = 1c9a46f7-bbb6-4393-a7c5-ae589f37de07 response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44
41542024-03-21T18:11:53.455ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
41552024-03-21T18:11:53.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41562024-03-21T18:11:53.455ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
41572024-03-21T18:11:53.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
41582024-03-21T18:11:53.455ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
41592024-03-21T18:11:53.455ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41602024-03-21T18:11:53.455ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
41612024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41622024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
41632024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
41642024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
41652024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41662024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
41672024-03-21T18:11:53.456ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
41682024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
41692024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
41702024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
41712024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
41722024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
41732024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
41742024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41752024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41762024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41772024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
41782024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41792024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41802024-03-21T18:11:53.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41812024-03-21T18:11:53.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41822024-03-21T18:11:53.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41832024-03-21T18:11:53.457ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4e50e7be-6fe9-4f8e-a8d0-2e9594b97494.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7c865eb6-b5e7-489f-b89d-276fb69b0134.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1742, tv_nsec: 181025267 } })
41842024-03-21T18:11:53.457ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40fdef96-95bd-4498-930b-07e0bc0ee2d0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7c261579-c0ac-44ad-8a5c-d19534e8de67.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7f65d905-b9ae-45c4-8202-b83c173f708b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1742, tv_nsec: 181130579 } })
41852024-03-21T18:11:53.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41862024-03-21T18:11:53.458ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41872024-03-21T18:11:53.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41882024-03-21T18:11:53.458ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
41892024-03-21T18:11:53.458ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.512805ms iteration = 18
41902024-03-21T18:11:53.458ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41912024-03-21T18:11:53.459ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
41922024-03-21T18:11:53.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.425001ms iteration = 10
41932024-03-21T18:11:53.459ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1742, tv_nsec: 182770368 } })
41942024-03-21T18:11:53.459ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41952024-03-21T18:11:53.461ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
41962024-03-21T18:11:53.461ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
41972024-03-21T18:11:53.461ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
41982024-03-21T18:11:53.461ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.05\u{b5}s iteration = 18
41992024-03-21T18:11:53.461ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
42002024-03-21T18:11:53.461ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.492579ms iteration = 10
42012024-03-21T18:11:53.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
42022024-03-21T18:11:53.462ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42032024-03-21T18:11:53.462ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
42042024-03-21T18:11:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42052024-03-21T18:11:53.462ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1742, tv_nsec: 186148269 } })
42062024-03-21T18:11:53.463ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
42072024-03-21T18:11:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42082024-03-21T18:11:53.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42092024-03-21T18:11:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
42102024-03-21T18:11:53.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42112024-03-21T18:11:53.464ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42122024-03-21T18:11:53.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42132024-03-21T18:11:53.464ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
42142024-03-21T18:11:53.464ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
42152024-03-21T18:11:53.464ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
42162024-03-21T18:11:53.464ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42172024-03-21T18:11:53.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42182024-03-21T18:11:53.465ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42192024-03-21T18:11:53.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42202024-03-21T18:11:53.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42212024-03-21T18:11:53.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42222024-03-21T18:11:53.465ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42232024-03-21T18:11:53.466ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
42242024-03-21T18:11:53.466ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
42252024-03-21T18:11:53.466ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
42262024-03-21T18:11:53.466ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42272024-03-21T18:11:53.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42282024-03-21T18:11:53.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42292024-03-21T18:11:53.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42302024-03-21T18:11:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
42312024-03-21T18:11:53.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42322024-03-21T18:11:53.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42332024-03-21T18:11:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
42342024-03-21T18:11:53.471ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42352024-03-21T18:11:53.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42362024-03-21T18:11:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f7e3fff9-b710-4667-b06d-53953d42fa9c, lookup_type: ById(f7e3fff9-b710-4667-b06d-53953d42fa9c) } result = Ok(())
42372024-03-21T18:11:53.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42382024-03-21T18:11:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42392024-03-21T18:11:53.472ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
42402024-03-21T18:11:53.472ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 0 version = 2
42412024-03-21T18:11:53.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42422024-03-21T18:11:53.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42432024-03-21T18:11:53.473ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42442024-03-21T18:11:53.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42452024-03-21T18:11:53.473ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42462024-03-21T18:11:53.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42472024-03-21T18:11:53.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42482024-03-21T18:11:53.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42492024-03-21T18:11:53.474ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42502024-03-21T18:11:53.475ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
42512024-03-21T18:11:53.475ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
42522024-03-21T18:11:53.475ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42532024-03-21T18:11:53.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42542024-03-21T18:11:53.476ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c original_time_created = 2024-03-21 18:03:42.846054 UTC target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c time_created = 2024-03-21 18:03:42.846054 UTC
42552024-03-21T18:11:53.476ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.159682ms iteration = 50
42562024-03-21T18:11:53.476ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42572024-03-21T18:11:53.477ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
42582024-03-21T18:11:53.477ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42592024-03-21T18:11:53.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
42602024-03-21T18:11:53.477ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42612024-03-21T18:11:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42622024-03-21T18:11:53.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42632024-03-21T18:11:53.479ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = df22a06a-09a0-44f9-9c1e-34991b189c7d dns_zone_name = oxide.test found_so_far = 1 version = 2
42642024-03-21T18:11:53.479ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
42652024-03-21T18:11:53.479ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-21 18:04:08.259984 UTC dns_group = external generation = 2
42662024-03-21T18:11:53.479ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.510701ms iteration = 10
42672024-03-21T18:11:53.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42682024-03-21T18:11:53.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42692024-03-21T18:11:53.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42702024-03-21T18:11:53.481ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e14dc16a-b53f-4164-9457-d2856b1d0e40 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
42712024-03-21T18:11:53.481ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
42722024-03-21T18:11:53.481ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-21 18:04:07.863813 UTC dns_group = internal generation = 1
42732024-03-21T18:11:53.481ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 26.061417ms iteration = 10
42742024-03-21T18:11:53.482ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
42752024-03-21T18:11:53.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
42762024-03-21T18:11:53.482ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:62 reason = blueprint disabled target_id = f7e3fff9-b710-4667-b06d-53953d42fa9c
42772024-03-21T18:11:53.482ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42782024-03-21T18:11:53.482ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 5.171\u{b5}s iteration = 11
42792024-03-21T18:11:53.482ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
42802024-03-21T18:11:53.482ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42812024-03-21T18:11:53.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42822024-03-21T18:11:53.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42832024-03-21T18:11:53.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42842024-03-21T18:11:53.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42852024-03-21T18:11:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
42862024-03-21T18:11:53.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42872024-03-21T18:11:53.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42882024-03-21T18:11:53.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
42892024-03-21T18:11:53.487ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42902024-03-21T18:11:53.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42912024-03-21T18:11:53.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1742, tv_nsec: 211051535 } })
42922024-03-21T18:11:53.488ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
42932024-03-21T18:11:53.489ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42942024-03-21T18:11:53.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42952024-03-21T18:11:53.495ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-21T18:03:53.561270Z, time_modified: 2024-03-21T18:03:53.561270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 6bddc419-7e94-416b-9f44-01287ffb5980, db_silo: Silo { identity: SiloIdentity { id: 6bddc419-7e94-416b-9f44-01287ffb5980, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-21T18:04:08.231292Z, time_modified: 2024-03-21T18:04:08.231292Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "f2a2edb9166e3600bb596ded32753aae90652877a3340c9d65f5b99219243cec" }] }) }
42962024-03-21T18:11:53.495ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 39.351925ms iteration = 10
42972024-03-21T18:11:53.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42982024-03-21T18:11:53.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42992024-03-21T18:11:53.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43002024-03-21T18:11:53.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43012024-03-21T18:11:53.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43022024-03-21T18:11:53.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7, lookup_type: ById(db83cea0-14d4-44e9-bc35-e1a0cfe2bcf7) } result = Ok(())
43032024-03-21T18:11:53.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43042024-03-21T18:11:53.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43052024-03-21T18:11:53.512ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 30.767111ms iteration = 18
43062024-03-21T18:11:53.599ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
43072024-03-21T18:11:53.618ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
43082024-03-21T18:11:53.618ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43092024-03-21T18:11:53.618ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e6d6608c-4d35-4a81-9842-96e5a8d5a12b", "content-length": "26", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43102024-03-21T18:11:53.618ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43112024-03-21T18:11:53.620ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43122024-03-21T18:11:53.639ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 zone_address = fd00:1122:3344:101::2
43132024-03-21T18:11:53.639ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43142024-03-21T18:11:53.640ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fa042a5e-fe43-47d5-9179-655c6c4ad29a", "content-length": "26", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43152024-03-21T18:11:53.640ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43162024-03-21T18:11:53.641ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
43172024-03-21T18:11:53.729ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
43182024-03-21T18:11:53.759ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
43192024-03-21T18:11:53.759ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43202024-03-21T18:11:53.759ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "607e8deb-f645-429d-ba31-08f86e890f16", "content-length": "26", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43212024-03-21T18:11:53.759ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43222024-03-21T18:11:53.760ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43232024-03-21T18:11:53.856ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
43242024-03-21T18:11:53.857ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4844c0d1-5313-4853-8807-b0878fc5d181", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
43252024-03-21T18:11:53.857ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
43262024-03-21T18:11:53.858ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "33fc2c19-b30a-45a6-9af7-5c02dee8fb47", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
43272024-03-21T18:11:53.858ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
43282024-03-21T18:11:53.859ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
43292024-03-21T18:11:53.859ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 403.777336ms iteration = 12
43302024-03-21T18:11:53.883ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 427.41626ms iteration = 18
43312024-03-21T18:11:53.975ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
43322024-03-21T18:11:54.001ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
43332024-03-21T18:11:54.001ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0494f937-cdb1-4b16-8994-a5e194d78b20", "content-length": "2", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43342024-03-21T18:11:54.002ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
43352024-03-21T18:11:54.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
43362024-03-21T18:11:54.004ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 548.408699ms iteration = 18
43372024-03-21T18:11:54.010ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43382024-03-21T18:11:54.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43392024-03-21T18:11:54.012ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:140 port = SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) } rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
43402024-03-21T18:11:54.012ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43412024-03-21T18:11:54.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43422024-03-21T18:11:54.018ZINFOnexus (BackgroundTasks): applying switch port config changes background_task = switch_port_config_manager changes = [(Switch0, SwitchPort { id: 5af745f9-4dd3-4b25-8f76-52b216f35d26, rack_id: 11ad3243-b088-4dcc-87e3-5a6072b1f137, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(ca00232c-d862-4687-99ac-9a3e1f883bd6) }, Apply(SwitchPortSettingsCombinedResult { settings: SwitchPortSettings { identity: SwitchPortSettingsIdentity { id: ca00232c-d862-4687-99ac-9a3e1f883bd6, name: Name(Name("default-uplink0")), description: "initial uplink configuration", time_created: 2024-03-21T18:03:56.604718Z, time_modified: 2024-03-21T18:03:56.604718Z, time_deleted: None } }, groups: [], port: SwitchPortConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, geometry: Qsfp28x1 }, links: [SwitchPortLinkConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, lldp_service_config_id: e3fdd158-770f-4b28-a73d-8b444f484257, link_name: "phy", mtu: SqlU16(1500), fec: None, speed: Speed40G, autoneg: false }], link_lldp: [LldpServiceConfig { id: e3fdd158-770f-4b28-a73d-8b444f484257, enabled: false, lldp_config_id: None }], interfaces: [], vlan_interfaces: [], routes: [SwitchPortRouteConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, interface_name: "phy0", dst: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), gw: V4(Ipv4Network { addr: 10.151.1.100, prefix: 32 }), vid: None }], bgp_peers: [], addresses: [SwitchPortAddressConfig { port_settings_id: ca00232c-d862-4687-99ac-9a3e1f883bd6, address_lot_block_id: 2da8830b-52b2-4aa4-9a32-05a5d1c93f3c, rsvd_address_lot_block_id: 663423af-f37d-45d8-b80e-44f746890c9b, address: V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 }), interface_name: "phy0" }] }))] file = nexus/src/app/background/sync_switch_configuration.rs:304 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
43432024-03-21T18:11:54.018ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
43442024-03-21T18:11:54.018ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7c1cd90a-b600-45a1-9ade-1a44b7e6c4dc", "content-length": "121", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43452024-03-21T18:11:54.018ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1347 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
43462024-03-21T18:11:54.018ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1374 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = ca00232c-d862-4687-99ac-9a3e1f883bd6
43472024-03-21T18:11:54.019ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:314 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
43482024-03-21T18:11:54.019ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:318 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {Switch0: {(10.151.1.100, Prefix4 { value: 0.0.0.0, length: 0 })}}
43492024-03-21T18:11:54.019ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:327 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
43502024-03-21T18:11:54.019ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:333 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 routes = {}
43512024-03-21T18:11:54.019ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:353 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137
43522024-03-21T18:11:54.019ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43532024-03-21T18:11:54.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43542024-03-21T18:11:54.021ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
43552024-03-21T18:11:54.021ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0c819ebd-7fbb-4a57-a04b-964327cea7a0", "content-length": "2", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43562024-03-21T18:11:54.021ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
43572024-03-21T18:11:54.022ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4f474b8d-5693-4033-9f93-64e049011346", "content-length": "2", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43582024-03-21T18:11:54.022ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:402 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch_location = Switch0
43592024-03-21T18:11:54.022ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
43602024-03-21T18:11:54.031ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
43612024-03-21T18:11:54.032ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4bec6362-4e48-4ac4-8379-52ddad4a48ef", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43622024-03-21T18:11:54.032ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43632024-03-21T18:11:54.033ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a9280bec-1d1a-4d5b-adb8-3e04a043bd5e", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43642024-03-21T18:11:54.033ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43652024-03-21T18:11:54.035ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "950ddb06-e284-410d-8e97-36ed913d5853", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43662024-03-21T18:11:54.035ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43672024-03-21T18:11:54.037ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
43682024-03-21T18:11:54.037ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 581.94088ms iteration = 12
43692024-03-21T18:11:54.067ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6c380556-04db-4d31-b951-7f11c0e08221", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43702024-03-21T18:11:54.067ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:578 rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 switch = Switch0
43712024-03-21T18:11:54.067ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
43722024-03-21T18:11:54.067ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 11ad3243-b088-4dcc-87e3-5a6072b1f137 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "03e0c8cb-254f-403c-b584-3c2bc075985e", "content-length": "443", "date": "Thu, 21 Mar 2024 18:11:53 GMT"} })
43732024-03-21T18:11:54.067ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43742024-03-21T18:11:54.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43752024-03-21T18:11:54.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43762024-03-21T18:11:54.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43772024-03-21T18:11:54.076ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 594.233251ms iteration = 18
43782024-03-21T18:11:54.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 703 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:43220 req_id = be2e58e7-d8f7-4958-a665-dad7d402c50d response_code = 200 uri = /metrics/collect/63f8ddb8-9337-4e6d-9183-6a3d2d46eb44