Max level shown:
12024-12-13T21:18:13.000ZDEBGtest_mgs_metrics (init_with_steps): Running step start_crdb
22024-12-13T21:18:13.000ZDEBGtest_mgs_metrics: Starting CRDB
32024-12-13T21:18:13.000ZINFOtest_mgs_metrics: cockroach temporary directory: /var/tmp/omicron_tmp/.tmpD6l1ct
42024-12-13T21:18:13.000ZINFOtest_mgs_metrics: cockroach: copying from seed tarball (/var/tmp/omicron_tmp/crdb-base-build/7b407944d72adf57b9399a944886614e49ac1a89225f6dcd880d61e7e086145e.tar) to storage directory (/var/tmp/omicron_tmp/.tmpD6l1ct/data)
52024-12-13T21:18:13.055ZINFOtest_mgs_metrics: cockroach command line: cockroach start-single-node --insecure --http-addr=:0 --store=path=/var/tmp/omicron_tmp/.tmpD6l1ct/data,ballast-size=0 --listen-addr [::1]:0 --listening-url-file /var/tmp/omicron_tmp/.tmpD6l1ct/listen-url
62024-12-13T21:18:13.055ZINFOtest_mgs_metrics: cockroach environment: BUILDOMAT_JOB_ID=01JF0W5NZMF0GRWE6JPVSFWHQC BUILDOMAT_TASK_ID=4 CARGO=/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/bin/cargo CARGO_HOME=/home/build/.cargo CARGO_INCREMENTAL=0 CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus CARGO_PKG_AUTHORS= CARGO_PKG_DESCRIPTION= CARGO_PKG_HOMEPAGE= CARGO_PKG_LICENSE=MPL-2.0 CARGO_PKG_LICENSE_FILE= CARGO_PKG_NAME=omicron-nexus CARGO_PKG_REPOSITORY= CARGO_PKG_RUST_VERSION= CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= CARGO_TERM_COLOR=always CI=true CRDB_SEED_TAR=/var/tmp/omicron_tmp/crdb-base-build/7b407944d72adf57b9399a944886614e49ac1a89225f6dcd880d61e7e086145e.tar GITHUB_BRANCH=sluggish-datasets GITHUB_REF=refs/heads/sluggish-datasets GITHUB_REPOSITORY=oxidecomputer/omicron GITHUB_SHA=b6506393d435740ac3a1fb86ea44770a762df51c GOTRACEBACK=crash HOME=/home/build LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-0605127ef2db4010/out:/work/oxidecomputer/omicron/target/debug/build/blake3-1e464eee91b26c77/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-70e33ff4b45d8dc0/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-6a6e82c218d03cb7/out/build:/work/oxidecomputer/omicron/target/debug/build/libz-sys-33d79b7b3a6a5533/out/lib:/work/oxidecomputer/omicron/target/debug/build/ring-84313340968b144b/out:/work/oxidecomputer/omicron/target/debug/build/ring-ae424a22a6f8daa6/out:/work/oxidecomputer/omicron/target/debug/build/tofino-b9675a1b9b2f961b/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib LOGNAME=build NEXTEST=1 NEXTEST_BIN_EXE_nexus=/work/oxidecomputer/omicron/target/debug/nexus NEXTEST_BIN_EXE_schema-updater=/work/oxidecomputer/omicron/target/debug/schema-updater NEXTEST_EXECUTION_MODE=process-per-test NEXTEST_LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-0605127ef2db4010/out:/work/oxidecomputer/omicron/target/debug/build/blake3-1e464eee91b26c77/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-70e33ff4b45d8dc0/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-6a6e82c218d03cb7/out/build:/work/oxidecomputer/omicron/target/debug/build/libz-sys-33d79b7b3a6a5533/out/lib:/work/oxidecomputer/omicron/target/debug/build/ring-84313340968b144b/out:/work/oxidecomputer/omicron/target/debug/build/ring-ae424a22a6f8daa6/out:/work/oxidecomputer/omicron/target/debug/build/tofino-b9675a1b9b2f961b/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib NEXTEST_RUN_ID=36091007-e070-4d37-8e83-02f01c5469c5 OUT_DIR=/work/oxidecomputer/omicron/target/debug/build/omicron-nexus-dfad632cd24b082a/out PATH=/work/oxidecomputer/omicron/out/mgd/root/opt/oxide/mgd/bin:/work/oxidecomputer/omicron/out/dendrite-stub/bin:/work/oxidecomputer/omicron/out/clickhouse:/work/oxidecomputer/omicron/out/cockroachdb/bin:/home/build/.cargo/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/ooce/bin:/opt/ooce/sbin PWD=/work/oxidecomputer/omicron RUSTC_BOOTSTRAP=1 RUSTDOCFLAGS=--document-private-items -D warnings RUSTFLAGS=-D warnings RUSTUP_HOME=/home/build/.rustup RUSTUP_TOOLCHAIN=1.81.0-x86_64-unknown-linux-gnu RUST_BACKTRACE=1 RUST_RECURSION_COUNT=1 SHLVL=1 SSL_CERT_DIR=/usr/lib/ssl/certs SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt TMPDIR=/var/tmp/omicron_tmp TZ=UTC USER=build __NEXTEST_ATTEMPT=1 __NEXTEST_ORIGINAL_CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus
72024-12-13T21:18:14.219ZINFOtest_mgs_metrics: cockroach pid: 157637
82024-12-13T21:18:14.221ZINFOtest_mgs_metrics: cockroach listen URL: postgresql://root@[::1]:45213/omicron?sslmode=disable
92024-12-13T21:18:14.221ZDEBGtest_mgs_metrics (init_with_steps): Running step start_clickhouse
102024-12-13T21:18:14.221ZDEBGtest_mgs_metrics: Starting Clickhouse
112024-12-13T21:18:14.890ZDEBGtest_mgs_metrics (init_with_steps): Running step start_gateway_switch0
122024-12-13T21:18:14.891ZDEBGtest_mgs_metrics: Starting Management Gateway
132024-12-13T21:18:15.022ZDEBGtest_mgs_metrics (init_with_steps): Running step start_gateway_switch1
142024-12-13T21:18:15.022ZDEBGtest_mgs_metrics: Starting Management Gateway
152024-12-13T21:18:15.150ZDEBGtest_mgs_metrics (init_with_steps): Running step start_dendrite_switch0
162024-12-13T21:18:15.156ZDEBGtest_mgs_metrics: Starting Dendrite for switch0
172024-12-13T21:18:15.194ZDEBGtest_mgs_metrics (init_with_steps): Running step start_dendrite_switch1
182024-12-13T21:18:15.195ZDEBGtest_mgs_metrics: Starting Dendrite for switch1
192024-12-13T21:18:15.218ZDEBGtest_mgs_metrics (init_with_steps): Running step start_mgd_switch0
202024-12-13T21:18:15.218ZDEBGtest_mgs_metrics: Starting mgd for switch0
212024-12-13T21:18:15.253ZDEBGtest_mgs_metrics: mgd port is 35889
222024-12-13T21:18:15.254ZDEBGtest_mgs_metrics (init_with_steps): Running step start_mgd_switch1
232024-12-13T21:18:15.254ZDEBGtest_mgs_metrics: Starting mgd for switch1
242024-12-13T21:18:15.312ZDEBGtest_mgs_metrics: mgd port is 44263
252024-12-13T21:18:15.313ZDEBGtest_mgs_metrics (init_with_steps): Running step record_switch_dns
262024-12-13T21:18:15.313ZDEBGtest_mgs_metrics: Recording DNS for the switch zones
272024-12-13T21:18:15.313ZDEBGtest_mgs_metrics (init_with_steps): Running step start_internal_dns
282024-12-13T21:18:15.313ZINFOtest_mgs_metrics (store): opening sled database path = /var/tmp/omicron_tmp/.tmprgkYGZ
292024-12-13T21:18:15.370ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
302024-12-13T21:18:15.370ZINFOtest_mgs_metrics (store): pruning trees for generations older than 0 keep = 3
312024-12-13T21:18:15.370ZINFOtest_mgs_metrics (dns): DNS server bound to address kind = dns local_address = [::1]:40206
322024-12-13T21:18:15.370ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:39439 method = GET path = /config versions = all
332024-12-13T21:18:15.370ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:39439 method = PUT path = /config versions = all
342024-12-13T21:18:15.370ZINFOtest_mgs_metrics (http): listening kind = dns local_addr = [::1]:39439
352024-12-13T21:18:15.370ZDEBGtest_mgs_metrics (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:39439
362024-12-13T21:18:15.370ZDEBGtest_mgs_metrics (init_with_steps): Running step start_external_dns
372024-12-13T21:18:15.370ZINFOtest_mgs_metrics (store): opening sled database path = /var/tmp/omicron_tmp/.tmppeGnNc
382024-12-13T21:18:15.371ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
392024-12-13T21:18:15.371ZINFOtest_mgs_metrics (store): pruning trees for generations older than 0 keep = 3
402024-12-13T21:18:15.371ZINFOtest_mgs_metrics (dns): DNS server bound to address kind = dns local_address = [::1]:37667
412024-12-13T21:18:15.371ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:41531 method = GET path = /config versions = all
422024-12-13T21:18:15.371ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:41531 method = PUT path = /config versions = all
432024-12-13T21:18:15.371ZINFOtest_mgs_metrics (http): listening kind = dns local_addr = [::1]:41531
442024-12-13T21:18:15.371ZDEBGtest_mgs_metrics (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:41531
452024-12-13T21:18:15.371ZDEBGtest_mgs_metrics (init_with_steps): Running step start_nexus_internal
462024-12-13T21:18:15.371ZDEBGtest_mgs_metrics: Starting Nexus (internal API)
472024-12-13T21:18:15.371ZINFOtest_mgs_metrics: setting up nexus server
482024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Action
492024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AnyActor
502024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AuthenticatedActor
512024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = BlueprintConfig
522024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Database
532024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DnsConfig
542024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Fleet
552024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Inventory
562024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IpPoolList
572024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ConsoleSessionList
582024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAuthRequestList
592024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloCertificateList
602024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloIdentityProviderList
612024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloUserList
622024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Project
632024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Disk
642024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Snapshot
652024-12-13T21:18:15.372ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ProjectImage
662024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Instance
672024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IpPool
682024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InstanceNetworkInterface
692024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Vpc
702024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = VpcRouter
712024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGateway
722024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGatewayIpPool
732024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGatewayIpAddress
742024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = RouterRoute
752024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = VpcSubnet
762024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = FloatingIp
772024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Image
782024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloImage
792024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AddressLot
802024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Blueprint
812024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = LoopbackAddress
822024-12-13T21:18:15.373ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Certificate
832024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ConsoleSession
842024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAuthRequest
852024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAccessToken
862024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = PhysicalDisk
872024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Rack
882024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = RoleBuiltin
892024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SshKey
902024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Silo
912024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloUser
922024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloGroup
932024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IdentityProvider
942024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SamlIdentityProvider
952024-12-13T21:18:15.374ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Sled
962024-12-13T21:18:15.375ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = TufRepo
972024-12-13T21:18:15.375ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = TufArtifact
982024-12-13T21:18:15.375ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Zpool
992024-12-13T21:18:15.375ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Service
1002024-12-13T21:18:15.375ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = UserBuiltin
1012024-12-13T21:18:15.375ZINFOtest_mgs_metrics (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
1022024-12-13T21:18:15.387ZINFOtest_mgs_metrics (ServerContext): Setting up resolver using DNS address: [::1]:40206
1032024-12-13T21:18:15.422ZINFOtest_mgs_metrics (DnsResolver): new DNS resolver addresses = [[::1]:40206]
1042024-12-13T21:18:15.422ZINFOtest_mgs_metrics (ServerContext): Setting up qorb database pool from a single host url = PostgresConfigWithUrl {\n url_raw: "postgresql://root@[::1]:45213/omicron?sslmode=disable",\n config: Config {\n user: Some(\n "root",\n ),\n password: None,\n dbname: Some(\n "omicron",\n ),\n options: None,\n application_name: None,\n ssl_mode: Disable,\n host: [\n Tcp(\n "::1",\n ),\n ],\n hostaddr: [],\n port: [\n 45213,\n ],\n connect_timeout: None,\n tcp_user_timeout: None,\n keepalives: true,\n keepalives_idle: 7200s,\n keepalives_interval: None,\n keepalives_retries: None,\n target_session_attrs: Any,\n channel_binding: Prefer,\n },\n}
1052024-12-13T21:18:15.422ZDEBGtest_mgs_metrics (ServerContext): registered USDT probes
1062024-12-13T21:18:15.444ZINFOtest_mgs_metrics (nexus): Database schema version is up to date desired_version = 116.0.0 found_version = 116.0.0
1072024-12-13T21:18:15.770ZDEBGtest_mgs_metrics (clickhouse-client): registered USDT probes id = 272f9fcc-700b-45ef-85a6-81a18631e24b
1082024-12-13T21:18:15.780ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /artifacts/{kind}/{name}/{version} versions = all
1092024-12-13T21:18:15.780ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /bgtasks versions = all
1102024-12-13T21:18:15.780ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /bgtasks/activate versions = all
1112024-12-13T21:18:15.780ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /bgtasks/view/{bgtask_name} versions = all
1122024-12-13T21:18:15.780ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /clickhouse/policy versions = all
1132024-12-13T21:18:15.780ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /clickhouse/policy versions = all
1142024-12-13T21:18:15.780ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
1152024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
1162024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
1172024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1182024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1192024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /demo-saga versions = all
1202024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1212024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /deployment/blueprints/all versions = all
1222024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1232024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1242024-12-13T21:18:15.781ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /deployment/blueprints/import versions = all
1252024-12-13T21:18:15.782ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /deployment/blueprints/regenerate versions = all
1262024-12-13T21:18:15.782ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /deployment/blueprints/target versions = all
1272024-12-13T21:18:15.782ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /deployment/blueprints/target versions = all
1282024-12-13T21:18:15.782ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = PUT path = /deployment/blueprints/target/enabled versions = all
1292024-12-13T21:18:15.782ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1302024-12-13T21:18:15.782ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = PUT path = /disks/{disk_id} versions = all
1312024-12-13T21:18:15.782ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /instances/{instance_id}/migrate versions = all
1322024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /metrics/collectors versions = all
1332024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1342024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /metrics/producers versions = all
1352024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1362024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /physical-disk/expunge versions = all
1372024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /probes/{sled} versions = all
1382024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1392024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /sagas versions = all
1402024-12-13T21:18:15.783ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /sagas/{saga_id} versions = all
1412024-12-13T21:18:15.785ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /sled-agents/{sled_id} versions = all
1422024-12-13T21:18:15.785ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /sled-agents/{sled_id} versions = all
1432024-12-13T21:18:15.785ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1442024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /sleds/add versions = all
1452024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /sleds/expunge versions = all
1462024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = GET path = /sleds/uninitialized versions = all
1472024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = PUT path = /switch/{switch_id} versions = all
1482024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = PUT path = /vmms/{propolis_id} versions = all
1492024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:44649 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1502024-12-13T21:18:15.786ZINFOtest_mgs_metrics (dropshot_internal): listening local_addr = 127.0.0.1:44649
1512024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (dropshot_internal): successfully registered DTrace USDT probes local_addr = 127.0.0.1:44649
1522024-12-13T21:18:15.786ZINFOtest_mgs_metrics (SEC): SEC running sec_id = 97039868-0386-49ec-9259-75c36edb1ac3
1532024-12-13T21:18:15.786ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1542024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1552024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1562024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in users
1572024-12-13T21:18:15.786ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1582024-12-13T21:18:15.786ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1592024-12-13T21:18:15.789ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1602024-12-13T21:18:15.800ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: a3 ad 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:58249 req_id = 05c985ad-9de0-478b-a829-23fe9ae37d8c
1612024-12-13T21:18:15.800ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41901,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58249 req_id = 05c985ad-9de0-478b-a829-23fe9ae37d8c
1622024-12-13T21:18:15.800ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41901, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_crucible-pantry._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 16, 95, 99, 114, 117, 99, 105, 98, 108, 101, 45, 112, 97, 110, 116, 114, 121, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_crucible-pantry._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58249 req_id = 05c985ad-9de0-478b-a829-23fe9ae37d8c
1632024-12-13T21:18:15.816ZINFOtest_mgs_metrics (DataLoader): created 5 built-in users
1642024-12-13T21:18:15.817ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1652024-12-13T21:18:15.817ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1662024-12-13T21:18:15.821ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1672024-12-13T21:18:15.823ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in roles
1682024-12-13T21:18:15.823ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1692024-12-13T21:18:15.823ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1702024-12-13T21:18:15.823ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1712024-12-13T21:18:15.861ZINFOtest_mgs_metrics (DataLoader): created 10 built-in roles
1722024-12-13T21:18:15.862ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1732024-12-13T21:18:15.862ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1742024-12-13T21:18:15.864ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1752024-12-13T21:18:15.865ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in role assignments
1762024-12-13T21:18:15.865ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1772024-12-13T21:18:15.865ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1782024-12-13T21:18:15.865ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1792024-12-13T21:18:16.605ZINFOtest_mgs_metrics (DataLoader): created 4 built-in role assignments
1802024-12-13T21:18:16.605ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1812024-12-13T21:18:16.605ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1822024-12-13T21:18:16.607ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1832024-12-13T21:18:16.608ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in silos
1842024-12-13T21:18:16.608ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1852024-12-13T21:18:16.608ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1862024-12-13T21:18:16.608ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1872024-12-13T21:18:16.725ZINFOtest_mgs_metrics (DataLoader): created 2 built-in silos
1882024-12-13T21:18:16.727ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1892024-12-13T21:18:16.727ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1902024-12-13T21:18:16.727ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1912024-12-13T21:18:16.781ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1922024-12-13T21:18:16.782ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1932024-12-13T21:18:16.783ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1942024-12-13T21:18:16.783ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in projects
1952024-12-13T21:18:16.783ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1962024-12-13T21:18:16.783ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1972024-12-13T21:18:16.784ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1982024-12-13T21:18:16.786ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
1992024-12-13T21:18:16.786ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2002024-12-13T21:18:16.786ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2012024-12-13T21:18:16.786ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2022024-12-13T21:18:16.786ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2032024-12-13T21:18:16.826ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2042024-12-13T21:18:16.826ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2052024-12-13T21:18:16.826ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2062024-12-13T21:18:16.826ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2072024-12-13T21:18:16.828ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2082024-12-13T21:18:16.832ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2092024-12-13T21:18:16.832ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2102024-12-13T21:18:16.832ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2112024-12-13T21:18:16.833ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2122024-12-13T21:18:16.833ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2132024-12-13T21:18:16.833ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2142024-12-13T21:18:16.835ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2152024-12-13T21:18:16.835ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2162024-12-13T21:18:16.836ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2172024-12-13T21:18:16.836ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2182024-12-13T21:18:16.836ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2192024-12-13T21:18:16.839ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2202024-12-13T21:18:16.839ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2212024-12-13T21:18:16.839ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2222024-12-13T21:18:16.840ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2232024-12-13T21:18:16.900ZINFOtest_mgs_metrics (DataLoader): created built-in services project
2242024-12-13T21:18:16.900ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2252024-12-13T21:18:16.900ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2262024-12-13T21:18:16.902ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2272024-12-13T21:18:16.903ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPCs
2282024-12-13T21:18:16.903ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2292024-12-13T21:18:16.903ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2302024-12-13T21:18:16.903ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2312024-12-13T21:18:16.922ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2322024-12-13T21:18:16.924ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2332024-12-13T21:18:16.924ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2342024-12-13T21:18:16.925ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2352024-12-13T21:18:16.926ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2362024-12-13T21:18:16.926ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2372024-12-13T21:18:16.926ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2382024-12-13T21:18:16.926ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2392024-12-13T21:18:16.928ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2402024-12-13T21:18:16.930ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2412024-12-13T21:18:16.930ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2422024-12-13T21:18:16.930ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2432024-12-13T21:18:16.930ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2442024-12-13T21:18:16.930ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2452024-12-13T21:18:16.930ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2462024-12-13T21:18:16.931ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2472024-12-13T21:18:16.932ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2482024-12-13T21:18:16.938ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2492024-12-13T21:18:16.938ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Project" using built-in user
2502024-12-13T21:18:16.938ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2512024-12-13T21:18:16.938ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2522024-12-13T21:18:16.938ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2532024-12-13T21:18:16.938ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2542024-12-13T21:18:16.938ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2552024-12-13T21:18:16.940ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2562024-12-13T21:18:16.941ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2572024-12-13T21:18:16.941ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2582024-12-13T21:18:16.941ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2592024-12-13T21:18:16.943ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2602024-12-13T21:18:16.943ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2612024-12-13T21:18:16.943ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2622024-12-13T21:18:16.943ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2632024-12-13T21:18:16.945ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2642024-12-13T21:18:16.949ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2652024-12-13T21:18:16.949ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2662024-12-13T21:18:16.949ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2672024-12-13T21:18:16.950ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2682024-12-13T21:18:17.003ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2692024-12-13T21:18:17.003ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2702024-12-13T21:18:17.003ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2712024-12-13T21:18:17.003ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2722024-12-13T21:18:17.004ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2732024-12-13T21:18:17.011ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2742024-12-13T21:18:17.011ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2752024-12-13T21:18:17.011ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2762024-12-13T21:18:17.012ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772024-12-13T21:18:17.017ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2782024-12-13T21:18:17.017ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2792024-12-13T21:18:17.017ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2802024-12-13T21:18:17.017ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2812024-12-13T21:18:17.019ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2822024-12-13T21:18:17.026ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2832024-12-13T21:18:17.026ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2842024-12-13T21:18:17.026ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2852024-12-13T21:18:17.027ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2862024-12-13T21:18:17.064ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2872024-12-13T21:18:17.064ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2882024-12-13T21:18:17.065ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2892024-12-13T21:18:17.096ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2902024-12-13T21:18:17.096ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2912024-12-13T21:18:17.097ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2922024-12-13T21:18:17.099ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2932024-12-13T21:18:17.099ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2942024-12-13T21:18:17.099ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2952024-12-13T21:18:17.099ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2962024-12-13T21:18:17.100ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2972024-12-13T21:18:17.102ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2982024-12-13T21:18:17.102ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2992024-12-13T21:18:17.102ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3002024-12-13T21:18:17.102ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3012024-12-13T21:18:17.103ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3022024-12-13T21:18:17.104ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3032024-12-13T21:18:17.104ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3042024-12-13T21:18:17.104ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3052024-12-13T21:18:17.105ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3062024-12-13T21:18:17.111ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3072024-12-13T21:18:17.111ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3082024-12-13T21:18:17.111ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3092024-12-13T21:18:17.111ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3102024-12-13T21:18:17.163ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3112024-12-13T21:18:17.164ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3122024-12-13T21:18:17.164ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3132024-12-13T21:18:17.164ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3142024-12-13T21:18:17.164ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152024-12-13T21:18:17.174ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3162024-12-13T21:18:17.174ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3172024-12-13T21:18:17.174ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3182024-12-13T21:18:17.174ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192024-12-13T21:18:17.175ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3202024-12-13T21:18:17.175ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3212024-12-13T21:18:17.175ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3222024-12-13T21:18:17.175ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3232024-12-13T21:18:17.175ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3242024-12-13T21:18:17.175ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3252024-12-13T21:18:17.176ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3262024-12-13T21:18:17.176ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3272024-12-13T21:18:17.176ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3282024-12-13T21:18:17.282ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3292024-12-13T21:18:17.282ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3302024-12-13T21:18:17.282ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3312024-12-13T21:18:17.282ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3322024-12-13T21:18:17.283ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332024-12-13T21:18:17.285ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3342024-12-13T21:18:17.285ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3352024-12-13T21:18:17.285ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3362024-12-13T21:18:17.286ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372024-12-13T21:18:17.289ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3382024-12-13T21:18:17.289ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3392024-12-13T21:18:17.289ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3402024-12-13T21:18:17.290ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3412024-12-13T21:18:17.290ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3422024-12-13T21:18:17.297ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3432024-12-13T21:18:17.297ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3442024-12-13T21:18:17.297ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3452024-12-13T21:18:17.300ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3462024-12-13T21:18:17.314ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPC firewall rules
3472024-12-13T21:18:17.315ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3482024-12-13T21:18:17.315ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3492024-12-13T21:18:17.315ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3502024-12-13T21:18:17.317ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3512024-12-13T21:18:17.317ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3522024-12-13T21:18:17.317ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3532024-12-13T21:18:17.319ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3542024-12-13T21:18:17.320ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3552024-12-13T21:18:17.321ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3562024-12-13T21:18:17.321ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3572024-12-13T21:18:17.321ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3582024-12-13T21:18:17.321ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3592024-12-13T21:18:17.321ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3602024-12-13T21:18:17.321ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3612024-12-13T21:18:17.323ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3622024-12-13T21:18:17.323ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3632024-12-13T21:18:17.323ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3642024-12-13T21:18:17.323ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3652024-12-13T21:18:17.324ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3662024-12-13T21:18:17.325ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3672024-12-13T21:18:17.325ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3682024-12-13T21:18:17.325ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3692024-12-13T21:18:17.327ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3702024-12-13T21:18:17.334ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3712024-12-13T21:18:17.334ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
3722024-12-13T21:18:17.334ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3732024-12-13T21:18:17.334ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3742024-12-13T21:18:17.334ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3752024-12-13T21:18:17.334ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3762024-12-13T21:18:17.335ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3772024-12-13T21:18:17.336ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3782024-12-13T21:18:17.337ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3792024-12-13T21:18:17.337ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3802024-12-13T21:18:17.337ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3812024-12-13T21:18:17.339ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3822024-12-13T21:18:17.339ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3832024-12-13T21:18:17.339ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3842024-12-13T21:18:17.340ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3852024-12-13T21:18:17.341ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3862024-12-13T21:18:17.348ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3872024-12-13T21:18:17.348ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3882024-12-13T21:18:17.348ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3892024-12-13T21:18:17.349ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3902024-12-13T21:18:17.389ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3912024-12-13T21:18:17.389ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3922024-12-13T21:18:17.389ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3932024-12-13T21:18:17.389ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3942024-12-13T21:18:17.390ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952024-12-13T21:18:17.391ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3962024-12-13T21:18:17.392ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3972024-12-13T21:18:17.392ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3982024-12-13T21:18:17.392ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992024-12-13T21:18:17.393ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4002024-12-13T21:18:17.394ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4012024-12-13T21:18:17.394ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4022024-12-13T21:18:17.394ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4032024-12-13T21:18:17.395ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4042024-12-13T21:18:17.400ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4052024-12-13T21:18:17.400ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4062024-12-13T21:18:17.400ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4072024-12-13T21:18:17.401ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4082024-12-13T21:18:17.487ZINFOtest_mgs_metrics (DataLoader): created built-in VPC firewall rules
4092024-12-13T21:18:17.487ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPC Subnets
4102024-12-13T21:18:17.487ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4112024-12-13T21:18:17.487ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4122024-12-13T21:18:17.488ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4132024-12-13T21:18:17.516ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4142024-12-13T21:18:17.518ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4152024-12-13T21:18:17.518ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4162024-12-13T21:18:17.522ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4172024-12-13T21:18:17.522ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4182024-12-13T21:18:17.523ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4192024-12-13T21:18:17.525ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4202024-12-13T21:18:17.526ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4212024-12-13T21:18:17.526ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4222024-12-13T21:18:17.526ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4232024-12-13T21:18:17.526ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4242024-12-13T21:18:17.529ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4252024-12-13T21:18:17.530ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4262024-12-13T21:18:17.530ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4272024-12-13T21:18:17.531ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4282024-12-13T21:18:17.533ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4292024-12-13T21:18:17.534ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4302024-12-13T21:18:17.534ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4312024-12-13T21:18:17.534ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4322024-12-13T21:18:17.536ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4332024-12-13T21:18:17.542ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4342024-12-13T21:18:17.542ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
4352024-12-13T21:18:17.542ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4362024-12-13T21:18:17.543ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4372024-12-13T21:18:17.543ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4382024-12-13T21:18:17.576ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4392024-12-13T21:18:17.576ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4402024-12-13T21:18:17.576ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4412024-12-13T21:18:17.576ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4422024-12-13T21:18:17.577ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4432024-12-13T21:18:17.579ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4442024-12-13T21:18:17.579ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4452024-12-13T21:18:17.579ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4462024-12-13T21:18:17.579ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4472024-12-13T21:18:17.581ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4482024-12-13T21:18:17.582ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4492024-12-13T21:18:17.582ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4502024-12-13T21:18:17.582ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4512024-12-13T21:18:17.583ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4522024-12-13T21:18:17.590ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4532024-12-13T21:18:17.614ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4542024-12-13T21:18:17.615ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4552024-12-13T21:18:17.616ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4562024-12-13T21:18:17.737ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4572024-12-13T21:18:17.737ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4582024-12-13T21:18:17.737ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4592024-12-13T21:18:17.746ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4602024-12-13T21:18:17.747ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4612024-12-13T21:18:17.747ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4622024-12-13T21:18:17.747ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4632024-12-13T21:18:17.747ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4642024-12-13T21:18:17.807ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4652024-12-13T21:18:17.809ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4662024-12-13T21:18:17.809ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4672024-12-13T21:18:17.809ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4682024-12-13T21:18:17.817ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4692024-12-13T21:18:17.818ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4702024-12-13T21:18:17.818ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4712024-12-13T21:18:17.818ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4722024-12-13T21:18:17.821ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4732024-12-13T21:18:17.827ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
4742024-12-13T21:18:17.830ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4752024-12-13T21:18:17.830ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4762024-12-13T21:18:17.830ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4772024-12-13T21:18:17.855ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4782024-12-13T21:18:17.855ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4792024-12-13T21:18:17.858ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4802024-12-13T21:18:17.874ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4812024-12-13T21:18:17.881ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4822024-12-13T21:18:17.885ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4832024-12-13T21:18:17.886ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4842024-12-13T21:18:17.886ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4852024-12-13T21:18:17.886ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4862024-12-13T21:18:17.886ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4872024-12-13T21:18:17.886ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4882024-12-13T21:18:17.886ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4892024-12-13T21:18:17.886ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4902024-12-13T21:18:17.886ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4912024-12-13T21:18:17.886ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4922024-12-13T21:18:17.886ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4932024-12-13T21:18:17.886ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4942024-12-13T21:18:17.891ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4952024-12-13T21:18:17.903ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4962024-12-13T21:18:17.903ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4972024-12-13T21:18:17.903ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4982024-12-13T21:18:17.950ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4992024-12-13T21:18:17.954ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5002024-12-13T21:18:17.954ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5012024-12-13T21:18:17.966ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5022024-12-13T21:18:17.969ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5032024-12-13T21:18:17.969ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5042024-12-13T21:18:17.969ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5052024-12-13T21:18:17.970ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5062024-12-13T21:18:17.970ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5072024-12-13T21:18:17.973ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5082024-12-13T21:18:17.973ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5092024-12-13T21:18:17.973ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5102024-12-13T21:18:17.974ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5112024-12-13T21:18:17.974ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5122024-12-13T21:18:17.974ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5132024-12-13T21:18:17.974ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5142024-12-13T21:18:17.974ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5152024-12-13T21:18:17.980ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5162024-12-13T21:18:17.981ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5172024-12-13T21:18:17.981ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5182024-12-13T21:18:17.981ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5192024-12-13T21:18:17.996ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5202024-12-13T21:18:17.996ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5212024-12-13T21:18:17.997ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5222024-12-13T21:18:18.009ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
5232024-12-13T21:18:18.014ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5242024-12-13T21:18:18.014ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5252024-12-13T21:18:18.014ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5262024-12-13T21:18:18.015ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5272024-12-13T21:18:18.034ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5282024-12-13T21:18:18.035ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5292024-12-13T21:18:18.035ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5302024-12-13T21:18:18.035ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5312024-12-13T21:18:18.047ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5322024-12-13T21:18:18.048ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5332024-12-13T21:18:18.049ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5342024-12-13T21:18:18.049ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5352024-12-13T21:18:18.055ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5362024-12-13T21:18:18.063ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5372024-12-13T21:18:18.077ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5382024-12-13T21:18:18.077ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5392024-12-13T21:18:18.078ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5402024-12-13T21:18:18.132ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5412024-12-13T21:18:18.135ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5422024-12-13T21:18:18.135ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5432024-12-13T21:18:18.142ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5442024-12-13T21:18:18.142ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5452024-12-13T21:18:18.144ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5462024-12-13T21:18:18.144ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5472024-12-13T21:18:18.144ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5482024-12-13T21:18:18.154ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5492024-12-13T21:18:18.156ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5502024-12-13T21:18:18.156ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5512024-12-13T21:18:18.156ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5522024-12-13T21:18:18.162ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5532024-12-13T21:18:18.164ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5542024-12-13T21:18:18.164ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5552024-12-13T21:18:18.164ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5562024-12-13T21:18:18.167ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5572024-12-13T21:18:18.175ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5582024-12-13T21:18:18.175ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5592024-12-13T21:18:18.175ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5602024-12-13T21:18:18.177ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5612024-12-13T21:18:18.186ZINFOtest_mgs_metrics (DataLoader): created built-in services vpc subnets
5622024-12-13T21:18:18.186ZINFOtest_mgs_metrics (DataLoader): created built-in services vpc
5632024-12-13T21:18:18.188ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5642024-12-13T21:18:18.188ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5652024-12-13T21:18:18.189ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5662024-12-13T21:18:18.189ZDEBGtest_mgs_metrics (DataLoader): attempting to create silo users
5672024-12-13T21:18:18.189ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5682024-12-13T21:18:18.189ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5692024-12-13T21:18:18.189ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5702024-12-13T21:18:18.240ZINFOtest_mgs_metrics (DataLoader): created 2 silo users
5712024-12-13T21:18:18.241ZDEBGtest_mgs_metrics (DataLoader): attempting to create silo user role assignments
5722024-12-13T21:18:18.241ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5732024-12-13T21:18:18.241ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5742024-12-13T21:18:18.241ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5752024-12-13T21:18:18.248ZINFOtest_mgs_metrics (DataLoader): created 2 silo user role assignments
5762024-12-13T21:18:18.249ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5772024-12-13T21:18:18.251ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5782024-12-13T21:18:18.251ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5792024-12-13T21:18:18.259ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5802024-12-13T21:18:18.259ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5812024-12-13T21:18:18.259ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5822024-12-13T21:18:18.287ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5832024-12-13T21:18:18.287ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5842024-12-13T21:18:18.287ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5852024-12-13T21:18:18.287ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5862024-12-13T21:18:18.287ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5872024-12-13T21:18:18.288ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5882024-12-13T21:18:18.292ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5892024-12-13T21:18:18.292ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5902024-12-13T21:18:18.292ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5912024-12-13T21:18:18.292ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5922024-12-13T21:18:18.333ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5932024-12-13T21:18:18.333ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5942024-12-13T21:18:18.333ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5952024-12-13T21:18:18.333ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5962024-12-13T21:18:18.334ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5972024-12-13T21:18:18.338ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5982024-12-13T21:18:18.338ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5992024-12-13T21:18:18.338ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6002024-12-13T21:18:18.338ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
6012024-12-13T21:18:18.338ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6022024-12-13T21:18:18.506ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6032024-12-13T21:18:18.508ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
6042024-12-13T21:18:18.509ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6052024-12-13T21:18:18.515ZINFOtest_mgs_metrics (nexus): populate complete
6062024-12-13T21:18:18.515ZINFOtest_mgs_metrics (nexus): activating background tasks
6072024-12-13T21:18:18.515ZDEBGtest_mgs_metrics (init_with_steps): Running step start_sled1
6082024-12-13T21:18:18.516ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6092024-12-13T21:18:18.569ZINFOtest_mgs_metrics (SledAgent): created simulated sled agent server = b6d65341-167c-41df-9b5c-41cded99c229 sim_mode = Explicit sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6102024-12-13T21:18:18.615ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /artifacts sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6112024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = DELETE path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6122024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6132024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6142024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6152024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6162024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6172024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /bootstore/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6182024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /cockroachdb sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6192024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6202024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6212024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /disks/{disk_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6222024-12-13T21:18:18.616ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /disks/{disk_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6232024-12-13T21:18:18.617ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /eip-gateways sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6242024-12-13T21:18:18.617ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /inventory sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6252024-12-13T21:18:18.622ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6262024-12-13T21:18:18.622ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6272024-12-13T21:18:18.622ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6282024-12-13T21:18:18.622ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6292024-12-13T21:18:18.622ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /omicron-zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6302024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /sled-identifiers sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6312024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /sled-role sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6322024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /sleds sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6332024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /support/dladm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6342024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /support/ipadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6352024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /support/zoneadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6362024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /support-bundles/{zpool_id}/{dataset_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6372024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = DELETE path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6382024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6392024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6402024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6412024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /switch-ports sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6422024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /timesync sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6432024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /update sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6442024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = DELETE path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6452024-12-13T21:18:18.623ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6462024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6472024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = DELETE path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6482024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6492024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6502024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6512024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6522024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /vmms/{propolis_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6532024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6542024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6552024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6562024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6572024-12-13T21:18:18.624ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6582024-12-13T21:18:18.625ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6592024-12-13T21:18:18.625ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6602024-12-13T21:18:18.625ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6612024-12-13T21:18:18.625ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /zones/bundle-cleanup sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6622024-12-13T21:18:18.625ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6632024-12-13T21:18:18.625ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = PUT path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6642024-12-13T21:18:18.625ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /zones/bundle-cleanup/utilization sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6652024-12-13T21:18:18.626ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /zones/bundles sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6662024-12-13T21:18:18.626ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6672024-12-13T21:18:18.626ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = POST path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6682024-12-13T21:18:18.626ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6692024-12-13T21:18:18.626ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6702024-12-13T21:18:18.626ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45365 method = GET path = /zpools sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6712024-12-13T21:18:18.626ZINFOtest_mgs_metrics (dropshot): listening local_addr = [::1]:45365 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6722024-12-13T21:18:18.626ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:45365 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6732024-12-13T21:18:18.627ZDEBGtest_mgs_metrics (omicron_sled_agent::sim::Server): contacting server nexus sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6742024-12-13T21:18:18.627ZDEBGtest_mgs_metrics (NexusClient): client request body = Some(Body) method = POST sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = http://127.0.0.1:44649/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
6752024-12-13T21:18:18.627ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:44649 remote_addr = 127.0.0.1:43002
6762024-12-13T21:18:18.627ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
6772024-12-13T21:18:18.627ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6782024-12-13T21:18:18.627ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6792024-12-13T21:18:18.627ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6802024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6812024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
6822024-12-13T21:18:18.628ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
6832024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
6842024-12-13T21:18:18.628ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config nor servers
6852024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 36.611\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6862024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
6872024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6882024-12-13T21:18:18.628ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
6892024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6902024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6912024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
6922024-12-13T21:18:18.628ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
6932024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
6942024-12-13T21:18:18.628ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config nor servers
6952024-12-13T21:18:18.628ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 22.94\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6962024-12-13T21:18:18.629ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
6972024-12-13T21:18:18.629ZINFOtest_mgs_metrics (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-13 21:08:18.619483767 UTC
6982024-12-13T21:18:18.629ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
6992024-12-13T21:18:18.629ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
7002024-12-13T21:18:18.629ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
7012024-12-13T21:18:18.630ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
7022024-12-13T21:18:18.630ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
7032024-12-13T21:18:18.630ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7042024-12-13T21:18:18.630ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
7052024-12-13T21:18:18.630ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7062024-12-13T21:18:18.630ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7072024-12-13T21:18:18.630ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
7082024-12-13T21:18:18.630ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
7092024-12-13T21:18:18.630ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7102024-12-13T21:18:18.630ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7112024-12-13T21:18:18.630ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
7122024-12-13T21:18:18.630ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
7132024-12-13T21:18:18.631ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7142024-12-13T21:18:18.631ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7152024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
7162024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
7172024-12-13T21:18:18.632ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
7182024-12-13T21:18:18.632ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7192024-12-13T21:18:18.632ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
7202024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7212024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7222024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
7232024-12-13T21:18:18.632ZWARNtest_mgs_metrics (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = no blueprint
7242024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 23.54\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7252024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
7262024-12-13T21:18:18.632ZWARNtest_mgs_metrics (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector reason = no blueprint
7272024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 11.14\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7282024-12-13T21:18:18.632ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
7292024-12-13T21:18:18.632ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
7302024-12-13T21:18:18.634ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7312024-12-13T21:18:18.634ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
7322024-12-13T21:18:18.634ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7332024-12-13T21:18:18.634ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7342024-12-13T21:18:18.634ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
7352024-12-13T21:18:18.634ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 897.409\u{b5}s iteration = 1 status = {"error":"task disabled"}
7362024-12-13T21:18:18.634ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
7372024-12-13T21:18:18.634ZINFOtest_mgs_metrics (BackgroundTasks): Decommissioned Disk Cleaner disabled background_task = decommissioned_disk_cleaner
7382024-12-13T21:18:18.634ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 24.59\u{b5}s iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7392024-12-13T21:18:18.634ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
7402024-12-13T21:18:18.634ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
7412024-12-13T21:18:18.634ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7422024-12-13T21:18:18.635ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
7432024-12-13T21:18:18.635ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7442024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7452024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
7462024-12-13T21:18:18.636ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
7472024-12-13T21:18:18.636ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7482024-12-13T21:18:18.636ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
7492024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7502024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7512024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
7522024-12-13T21:18:18.636ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
7532024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7542024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7552024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
7562024-12-13T21:18:18.636ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
7572024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7582024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7592024-12-13T21:18:18.636ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
7602024-12-13T21:18:18.636ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
7612024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7622024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7632024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
7642024-12-13T21:18:18.637ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
7652024-12-13T21:18:18.637ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7662024-12-13T21:18:18.637ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
7672024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7682024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7692024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
7702024-12-13T21:18:18.637ZINFOtest_mgs_metrics (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
7712024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_updater elapsed = 93.812\u{b5}s iteration = 1 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7722024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
7732024-12-13T21:18:18.637ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
7742024-12-13T21:18:18.637ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7752024-12-13T21:18:18.638ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7762024-12-13T21:18:18.638ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
7772024-12-13T21:18:18.639ZDEBGtest_mgs_metrics (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7782024-12-13T21:18:18.639ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
7792024-12-13T21:18:18.639ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7802024-12-13T21:18:18.639ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7812024-12-13T21:18:18.639ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
7822024-12-13T21:18:18.639ZINFOtest_mgs_metrics (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
7832024-12-13T21:18:18.639ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
7842024-12-13T21:18:18.639ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7852024-12-13T21:18:18.639ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
7862024-12-13T21:18:18.639ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7872024-12-13T21:18:18.641ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7882024-12-13T21:18:18.641ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
7892024-12-13T21:18:18.641ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
7902024-12-13T21:18:18.641ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7912024-12-13T21:18:18.641ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7922024-12-13T21:18:18.642ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
7932024-12-13T21:18:18.643ZDEBGtest_mgs_metrics (SagaRecoverer): listing candidate sagas for recovery
7942024-12-13T21:18:18.643ZTRACtest_mgs_metrics (SagaRecoverer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database
7952024-12-13T21:18:18.643ZDEBGtest_mgs_metrics (SagaRecoverer): roles roles = RoleSet { roles: {} }
7962024-12-13T21:18:18.643ZDEBGtest_mgs_metrics (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
7972024-12-13T21:18:18.643ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
7982024-12-13T21:18:18.643ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
7992024-12-13T21:18:18.643ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
8002024-12-13T21:18:18.643ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
8012024-12-13T21:18:18.643ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
8022024-12-13T21:18:18.643ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
8032024-12-13T21:18:18.643ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8042024-12-13T21:18:18.644ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8052024-12-13T21:18:18.645ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
8062024-12-13T21:18:18.645ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database
8072024-12-13T21:18:18.657ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
8082024-12-13T21:18:18.660ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8092024-12-13T21:18:18.660ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
8102024-12-13T21:18:18.661ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
8112024-12-13T21:18:18.661ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8122024-12-13T21:18:18.661ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8132024-12-13T21:18:18.662ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
8142024-12-13T21:18:18.662ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database
8152024-12-13T21:18:18.662ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
8162024-12-13T21:18:18.664ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8172024-12-13T21:18:18.664ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43002 req_id = c3522d80-6bab-4e6d-843f-34294b455258 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8182024-12-13T21:18:18.665ZINFOtest_mgs_metrics (nexus): registered sled agent sled_uuid = b6d65341-167c-41df-9b5c-41cded99c229
8192024-12-13T21:18:18.665ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
8202024-12-13T21:18:18.665ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8212024-12-13T21:18:18.665ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
8222024-12-13T21:18:18.665ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8232024-12-13T21:18:18.666ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8242024-12-13T21:18:18.666ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: bf ca 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 ............._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:49377 req_id = 8c6cc21b-7605-419d-bb4c-16949fefafb5
8252024-12-13T21:18:18.666ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49098,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49377 req_id = 8c6cc21b-7605-419d-bb4c-16949fefafb5
8262024-12-13T21:18:18.666ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 49098, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_nameservice._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 12, 95, 110, 97, 109, 101, 115, 101, 114, 118, 105, 99, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_nameservice._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49377 req_id = 8c6cc21b-7605-419d-bb4c-16949fefafb5
8272024-12-13T21:18:18.666ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: b1 6c 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 .l..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51317 req_id = 90cd6eba-41eb-4175-90a9-20f90c4a2499
8282024-12-13T21:18:18.666ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 45420,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51317 req_id = 90cd6eba-41eb-4175-90a9-20f90c4a2499
8292024-12-13T21:18:18.667ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45420, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_external-dns._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 13, 95, 101, 120, 116, 101, 114, 110, 97, 108, 45, 100, 110, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_external-dns._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51317 req_id = 90cd6eba-41eb-4175-90a9-20f90c4a2499
8302024-12-13T21:18:18.667ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
8312024-12-13T21:18:18.667ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8322024-12-13T21:18:18.667ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
8332024-12-13T21:18:18.668ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8342024-12-13T21:18:18.668ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8352024-12-13T21:18:18.668ZWARNtest_mgs_metrics (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_internal dns_group = internal error = no record found for Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8362024-12-13T21:18:18.668ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 49.783823ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_nameservice._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8372024-12-13T21:18:18.668ZWARNtest_mgs_metrics (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_external dns_group = external error = no record found for Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8382024-12-13T21:18:18.668ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 49.020408ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_external-dns._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8392024-12-13T21:18:18.685ZTRACtest_mgs_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8402024-12-13T21:18:18.688ZTRACtest_mgs_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8412024-12-13T21:18:18.716ZINFOtest_mgs_metrics (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-13 21:08:18.619483767 UTC pruned = {}
8422024-12-13T21:18:18.716ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 97.094606ms iteration = 1 status = {"expiration":"2024-12-13T21:08:18.619483767Z","pruned":[]}
8432024-12-13T21:18:18.716ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 39 0e 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 9............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:56302 req_id = b7f9d285-a7f0-472e-970e-0f1ff622deb4
8442024-12-13T21:18:18.718ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14606,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56302 req_id = b7f9d285-a7f0-472e-970e-0f1ff622deb4
8452024-12-13T21:18:18.718ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14606, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56302 req_id = b7f9d285-a7f0-472e-970e-0f1ff622deb4
8462024-12-13T21:18:18.718ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: fc 7f 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:58535 req_id = 5398a3eb-2fc7-4e74-a049-2562921020f0
8472024-12-13T21:18:18.718ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64639,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58535 req_id = 5398a3eb-2fc7-4e74-a049-2562921020f0
8482024-12-13T21:18:18.718ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 64639, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58535 req_id = 5398a3eb-2fc7-4e74-a049-2562921020f0
8492024-12-13T21:18:18.719ZERROtest_mgs_metrics (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8502024-12-13T21:18:18.723ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 98.522206ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8512024-12-13T21:18:18.725ZERROtest_mgs_metrics (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8522024-12-13T21:18:18.725ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 98.183049ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8532024-12-13T21:18:18.815ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8542024-12-13T21:18:18.824ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8552024-12-13T21:18:18.826ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
8562024-12-13T21:18:18.826ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8572024-12-13T21:18:18.826ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8582024-12-13T21:18:18.831ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 210.19301ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8592024-12-13T21:18:18.843ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
8602024-12-13T21:18:18.843ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8612024-12-13T21:18:18.844ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8622024-12-13T21:18:18.991ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
8632024-12-13T21:18:18.991ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8642024-12-13T21:18:18.992ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8652024-12-13T21:18:19.014ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 371.71855ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
8662024-12-13T21:18:19.106ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8672024-12-13T21:18:19.118ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8682024-12-13T21:18:19.118ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
8692024-12-13T21:18:19.118ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8702024-12-13T21:18:19.119ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8712024-12-13T21:18:19.119ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8722024-12-13T21:18:19.130ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8732024-12-13T21:18:19.142ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
8742024-12-13T21:18:19.142ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8752024-12-13T21:18:19.143ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8762024-12-13T21:18:19.154ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8772024-12-13T21:18:19.162ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8782024-12-13T21:18:19.162ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
8792024-12-13T21:18:19.162ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8802024-12-13T21:18:19.169ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8812024-12-13T21:18:19.237ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8822024-12-13T21:18:19.280ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8832024-12-13T21:18:19.280ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
8842024-12-13T21:18:19.280ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8852024-12-13T21:18:19.284ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8862024-12-13T21:18:19.289ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8872024-12-13T21:18:19.305ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8882024-12-13T21:18:19.305ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
8892024-12-13T21:18:19.305ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8902024-12-13T21:18:19.316ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8912024-12-13T21:18:19.329ZDEBGtest_mgs_metrics (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
8922024-12-13T21:18:19.335ZINFOtest_mgs_metrics (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
8932024-12-13T21:18:19.335ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 694.205799ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
8942024-12-13T21:18:19.336ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
8952024-12-13T21:18:19.336ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8962024-12-13T21:18:19.336ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8972024-12-13T21:18:19.337ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
8982024-12-13T21:18:19.338ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8992024-12-13T21:18:19.338ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9002024-12-13T21:18:19.338ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
9012024-12-13T21:18:19.338ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9022024-12-13T21:18:19.339ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9032024-12-13T21:18:19.340ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9042024-12-13T21:18:19.346ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9052024-12-13T21:18:19.346ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
9062024-12-13T21:18:19.346ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9072024-12-13T21:18:19.347ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9082024-12-13T21:18:19.347ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9092024-12-13T21:18:19.347ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9102024-12-13T21:18:19.347ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 704.008607ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
9112024-12-13T21:18:19.348ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9122024-12-13T21:18:19.364ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9132024-12-13T21:18:19.369ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9142024-12-13T21:18:19.379ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9152024-12-13T21:18:19.395ZTRACtest_mgs_metrics (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = instance failed total_found = 0
9162024-12-13T21:18:19.396ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
9172024-12-13T21:18:19.396ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9182024-12-13T21:18:19.406ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9192024-12-13T21:18:19.408ZINFOtest_mgs_metrics (SagaRecoverer): listed in-progress sagas count = 0
9202024-12-13T21:18:19.408ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 766.453989ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
9212024-12-13T21:18:19.417ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9222024-12-13T21:18:19.417ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9232024-12-13T21:18:19.417ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9242024-12-13T21:18:19.418ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 756.001168ms iteration = 1 status = {"errors":[],"records_set_to_done":[]}
9252024-12-13T21:18:19.446ZWARNtest_mgs_metrics (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set
9262024-12-13T21:18:19.446ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 825.243334ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
9272024-12-13T21:18:19.472ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
9282024-12-13T21:18:19.472ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9292024-12-13T21:18:19.472ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9302024-12-13T21:18:19.472ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9312024-12-13T21:18:19.473ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9322024-12-13T21:18:19.494ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 867.1571ms iteration = 1 status = {}
9332024-12-13T21:18:19.502ZWARNtest_mgs_metrics (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0
9342024-12-13T21:18:19.502ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 883.501017ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
9352024-12-13T21:18:19.584ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
9362024-12-13T21:18:19.584ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
9372024-12-13T21:18:19.584ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9382024-12-13T21:18:19.584ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9392024-12-13T21:18:19.584ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9402024-12-13T21:18:19.589ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
9412024-12-13T21:18:19.589ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9422024-12-13T21:18:19.589ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9432024-12-13T21:18:19.595ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
9442024-12-13T21:18:19.595ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9452024-12-13T21:18:19.595ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9462024-12-13T21:18:19.595ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9472024-12-13T21:18:19.595ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9482024-12-13T21:18:19.601ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9492024-12-13T21:18:19.601ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9502024-12-13T21:18:19.601ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9512024-12-13T21:18:19.607ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9522024-12-13T21:18:19.608ZWARNtest_mgs_metrics (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0
9532024-12-13T21:18:19.608ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 990.758538ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
9542024-12-13T21:18:19.643ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker
9552024-12-13T21:18:19.643ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 1.016781449s iteration = 1 status = {"error":"inventory collection is None"}
9562024-12-13T21:18:19.690ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9572024-12-13T21:18:19.690ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9582024-12-13T21:18:19.690ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9592024-12-13T21:18:19.690ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9602024-12-13T21:18:19.690ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9612024-12-13T21:18:19.690ZINFOtest_mgs_metrics (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9622024-12-13T21:18:19.690ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 1.054975368s iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
9632024-12-13T21:18:19.718ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 1.074999211s iteration = 1 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
9642024-12-13T21:18:19.736ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
9652024-12-13T21:18:19.736ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
9662024-12-13T21:18:19.737ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a3 08 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55208 req_id = 8d108d0e-b7e1-4a5e-b62e-a2d39855876f
9672024-12-13T21:18:19.737ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41736,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55208 req_id = 8d108d0e-b7e1-4a5e-b62e-a2d39855876f
9682024-12-13T21:18:19.737ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41736, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55208 req_id = 8d108d0e-b7e1-4a5e-b62e-a2d39855876f
9692024-12-13T21:18:19.738ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
9702024-12-13T21:18:19.738ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.115941429s iteration = 1 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
9712024-12-13T21:18:19.750ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
9722024-12-13T21:18:19.750ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
9732024-12-13T21:18:19.750ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9742024-12-13T21:18:19.750ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
9752024-12-13T21:18:19.750ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9762024-12-13T21:18:19.750ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9772024-12-13T21:18:19.750ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 1083067 local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43002 req_id = c3522d80-6bab-4e6d-843f-34294b455258 response_code = 204 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
9782024-12-13T21:18:19.756ZDEBGtest_mgs_metrics (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:44649/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229", status: 204, headers: {"x-request-id": "c3522d80-6bab-4e6d-843f-34294b455258", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} }) sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9792024-12-13T21:18:19.756ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9802024-12-13T21:18:19.756ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9812024-12-13T21:18:19.756ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9822024-12-13T21:18:19.756ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9832024-12-13T21:18:19.756ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9842024-12-13T21:18:19.756ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9852024-12-13T21:18:19.757ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9862024-12-13T21:18:19.757ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9872024-12-13T21:18:19.757ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46167 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9882024-12-13T21:18:19.757ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:46167 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9892024-12-13T21:18:19.757ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:46167 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9902024-12-13T21:18:19.757ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:46167 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9912024-12-13T21:18:19.757ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9922024-12-13T21:18:19.757ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9932024-12-13T21:18:19.757ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9942024-12-13T21:18:19.758ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9952024-12-13T21:18:19.758ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9962024-12-13T21:18:19.758ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9972024-12-13T21:18:19.758ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9982024-12-13T21:18:19.758ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9992024-12-13T21:18:19.758ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42403 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10002024-12-13T21:18:19.759ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:42403 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10012024-12-13T21:18:19.759ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:42403 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10022024-12-13T21:18:19.759ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:42403 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10032024-12-13T21:18:19.759ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10042024-12-13T21:18:19.759ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10052024-12-13T21:18:19.759ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10062024-12-13T21:18:19.759ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10072024-12-13T21:18:19.759ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10082024-12-13T21:18:19.759ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10092024-12-13T21:18:19.759ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10102024-12-13T21:18:19.760ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10112024-12-13T21:18:19.760ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39879 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10122024-12-13T21:18:19.760ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:39879 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10132024-12-13T21:18:19.760ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:39879 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10142024-12-13T21:18:19.760ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:39879 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10152024-12-13T21:18:19.760ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10162024-12-13T21:18:19.760ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10172024-12-13T21:18:19.760ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10182024-12-13T21:18:19.760ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10192024-12-13T21:18:19.760ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10202024-12-13T21:18:19.761ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10212024-12-13T21:18:19.761ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10222024-12-13T21:18:19.761ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10232024-12-13T21:18:19.761ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34363 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10242024-12-13T21:18:19.762ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:34363 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10252024-12-13T21:18:19.762ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:34363 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10262024-12-13T21:18:19.762ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:34363 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10272024-12-13T21:18:19.762ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10282024-12-13T21:18:19.762ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10292024-12-13T21:18:19.763ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10302024-12-13T21:18:19.763ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10312024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10322024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10332024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10342024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10352024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37723 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10362024-12-13T21:18:19.764ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:37723 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10372024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:37723 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10382024-12-13T21:18:19.764ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:37723 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10392024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10402024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10412024-12-13T21:18:19.764ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10422024-12-13T21:18:19.765ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10432024-12-13T21:18:19.765ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10442024-12-13T21:18:19.765ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10452024-12-13T21:18:19.765ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10462024-12-13T21:18:19.766ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10472024-12-13T21:18:19.766ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44931 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10482024-12-13T21:18:19.766ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:44931 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10492024-12-13T21:18:19.766ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:44931 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10502024-12-13T21:18:19.766ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:44931 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10512024-12-13T21:18:19.766ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10522024-12-13T21:18:19.766ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10532024-12-13T21:18:19.767ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10542024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10552024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10562024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10572024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10582024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10592024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39991 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10602024-12-13T21:18:19.768ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:39991 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10612024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:39991 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10622024-12-13T21:18:19.768ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:39991 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10632024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10642024-12-13T21:18:19.768ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10652024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10662024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10672024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10682024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10692024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10702024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10712024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39259 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10722024-12-13T21:18:19.769ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:39259 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10732024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:39259 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10742024-12-13T21:18:19.769ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:39259 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10752024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10762024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10772024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10782024-12-13T21:18:19.769ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10792024-12-13T21:18:19.771ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10802024-12-13T21:18:19.771ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10812024-12-13T21:18:19.771ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10822024-12-13T21:18:19.771ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10832024-12-13T21:18:19.771ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38675 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10842024-12-13T21:18:19.771ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38675 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10852024-12-13T21:18:19.771ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38675 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10862024-12-13T21:18:19.774ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38675 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10872024-12-13T21:18:19.774ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10882024-12-13T21:18:19.774ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10892024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10902024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10912024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10922024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10932024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10942024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10952024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10962024-12-13T21:18:19.776ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45607 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10972024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45607 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10982024-12-13T21:18:19.776ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45607 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10992024-12-13T21:18:19.776ZDEBGtest_mgs_metrics (init_with_steps): Running step start_sled2
11002024-12-13T21:18:19.776ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11012024-12-13T21:18:19.898ZINFOtest_mgs_metrics (SledAgent): created simulated sled agent server = 039be560-54cc-49e3-88df-1a29dadbf913 sim_mode = Explicit sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11022024-12-13T21:18:19.989ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /artifacts sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11032024-12-13T21:18:19.992ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = DELETE path = /artifacts/{sha256} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11042024-12-13T21:18:19.992ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /artifacts/{sha256} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11052024-12-13T21:18:19.992ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11062024-12-13T21:18:19.992ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11072024-12-13T21:18:19.992ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11082024-12-13T21:18:19.992ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11092024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /bootstore/status sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11102024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /cockroachdb sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11112024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /datasets sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11122024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /datasets sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11132024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /disks/{disk_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11142024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /disks/{disk_id}/poke sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11152024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /eip-gateways sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11162024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /inventory sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11172024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /network-bootstore-config sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11182024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /network-bootstore-config sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11192024-12-13T21:18:19.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /omicron-physical-disks sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11202024-12-13T21:18:19.994ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /omicron-physical-disks sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11212024-12-13T21:18:19.994ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /omicron-zones sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11222024-12-13T21:18:19.994ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /sled-identifiers sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11232024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /sled-role sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11242024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /sleds sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11252024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /support/dladm-info sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11262024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /support/ipadm-info sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11272024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /support/zoneadm-info sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11282024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /support-bundles/{zpool_id}/{dataset_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11292024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = DELETE path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11302024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11312024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11322024-12-13T21:18:19.995ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11332024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /switch-ports sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11342024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /timesync sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11352024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /update sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11362024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = DELETE path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11372024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11382024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11392024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = DELETE path = /vmms/{propolis_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11402024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /vmms/{propolis_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11412024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11422024-12-13T21:18:19.996ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11432024-12-13T21:18:19.997ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11442024-12-13T21:18:19.997ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /vmms/{propolis_id}/poke sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11452024-12-13T21:18:19.997ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11462024-12-13T21:18:19.997ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11472024-12-13T21:18:19.997ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /vmms/{propolis_id}/state sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11482024-12-13T21:18:19.997ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /vmms/{propolis_id}/state sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11492024-12-13T21:18:19.997ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11502024-12-13T21:18:19.998ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /vpc-routes sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11512024-12-13T21:18:19.998ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /vpc-routes sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11522024-12-13T21:18:19.998ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /zones sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11532024-12-13T21:18:19.998ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /zones/bundle-cleanup sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11542024-12-13T21:18:19.998ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /zones/bundle-cleanup/context sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11552024-12-13T21:18:19.998ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = PUT path = /zones/bundle-cleanup/context sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11562024-12-13T21:18:19.998ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /zones/bundle-cleanup/utilization sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11572024-12-13T21:18:19.998ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /zones/bundles sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11582024-12-13T21:18:19.999ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /zones/bundles/{zone_name} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11592024-12-13T21:18:19.999ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = POST path = /zones/bundles/{zone_name} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11602024-12-13T21:18:19.999ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11612024-12-13T21:18:19.999ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11622024-12-13T21:18:19.999ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:42715 method = GET path = /zpools sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11632024-12-13T21:18:20.000ZINFOtest_mgs_metrics (dropshot): listening local_addr = [::1]:42715 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11642024-12-13T21:18:20.000ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:42715 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11652024-12-13T21:18:20.000ZDEBGtest_mgs_metrics (omicron_sled_agent::sim::Server): contacting server nexus sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11662024-12-13T21:18:20.000ZDEBGtest_mgs_metrics (NexusClient): client request body = Some(Body) method = POST sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = http://127.0.0.1:44649/sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
11672024-12-13T21:18:20.001ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 1.359674829s iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11682024-12-13T21:18:20.001ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
11692024-12-13T21:18:20.001ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11702024-12-13T21:18:20.001ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11712024-12-13T21:18:20.001ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_replacement elapsed = 1.361839015s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11722024-12-13T21:18:20.002ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11732024-12-13T21:18:20.002ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11742024-12-13T21:18:20.002ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11752024-12-13T21:18:20.002ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11762024-12-13T21:18:20.002ZWARNtest_mgs_metrics (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone
11772024-12-13T21:18:20.002ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 1.376167528s iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
11782024-12-13T21:18:20.002ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:44649 remote_addr = 127.0.0.1:43004
11792024-12-13T21:18:20.002ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43004 req_id = 00f90631-d375-41f9-b885-b7f8a5e8d084 uri = /sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
11802024-12-13T21:18:20.002ZINFOtest_mgs_metrics (nexus): registered sled agent sled_uuid = 039be560-54cc-49e3-88df-1a29dadbf913
11812024-12-13T21:18:20.021ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11822024-12-13T21:18:20.037ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11832024-12-13T21:18:20.037ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
11842024-12-13T21:18:20.038ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11852024-12-13T21:18:20.038ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11862024-12-13T21:18:20.087ZTRACtest_mgs_metrics (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = start saga failed total_found = 0
11872024-12-13T21:18:20.087ZWARNtest_mgs_metrics (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
11882024-12-13T21:18:20.088ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 1.4522612s iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
11892024-12-13T21:18:20.216ZINFOtest_mgs_metrics (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
11902024-12-13T21:18:20.217ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:45365/vpc-routes
11912024-12-13T21:18:20.219ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
11922024-12-13T21:18:20.219ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
11932024-12-13T21:18:20.219ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11942024-12-13T21:18:20.233ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11952024-12-13T21:18:20.235ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
11962024-12-13T21:18:20.235ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
11972024-12-13T21:18:20.235ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
11982024-12-13T21:18:20.236ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11992024-12-13T21:18:20.236ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12002024-12-13T21:18:20.238ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12012024-12-13T21:18:20.239ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 1.578574214s iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
12022024-12-13T21:18:20.240ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
12032024-12-13T21:18:20.240ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12042024-12-13T21:18:20.240ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
12052024-12-13T21:18:20.240ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12062024-12-13T21:18:20.242ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12072024-12-13T21:18:20.244ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12082024-12-13T21:18:20.244ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12092024-12-13T21:18:20.244ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12102024-12-13T21:18:20.244ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12112024-12-13T21:18:20.245ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45365 remote_addr = [::1]:41848 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12122024-12-13T21:18:20.249ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45365 method = GET remote_addr = [::1]:41848 req_id = 5af78fff-96c7-40a7-94a3-282de3299ccd sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
12132024-12-13T21:18:20.250ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 8e 9e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51469 req_id = f8bec27e-d51a-4e6b-8639-eddfc9a6cbdf
12142024-12-13T21:18:20.251ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36510,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51469 req_id = f8bec27e-d51a-4e6b-8639-eddfc9a6cbdf
12152024-12-13T21:18:20.252ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 36510, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51469 req_id = f8bec27e-d51a-4e6b-8639-eddfc9a6cbdf
12162024-12-13T21:18:20.275ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 25444 local_addr = [::1]:45365 method = GET remote_addr = [::1]:41848 req_id = 5af78fff-96c7-40a7-94a3-282de3299ccd response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
12172024-12-13T21:18:20.277ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12182024-12-13T21:18:20.280ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12192024-12-13T21:18:20.281ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
12202024-12-13T21:18:20.281ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12212024-12-13T21:18:20.281ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
12222024-12-13T21:18:20.281ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12232024-12-13T21:18:20.281ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12242024-12-13T21:18:20.283ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 285825 local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43004 req_id = 00f90631-d375-41f9-b885-b7f8a5e8d084 response_code = 204 uri = /sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
12252024-12-13T21:18:20.285ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
12262024-12-13T21:18:20.285ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 539.147005ms iteration = 2 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
12272024-12-13T21:18:20.285ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Signaled
12282024-12-13T21:18:20.285ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
12292024-12-13T21:18:20.286ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12302024-12-13T21:18:20.286ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
12312024-12-13T21:18:20.286ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
12322024-12-13T21:18:20.287ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
12332024-12-13T21:18:20.287ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:45365/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5af78fff-96c7-40a7-94a3-282de3299ccd", "content-length": "2", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
12342024-12-13T21:18:20.288ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
12352024-12-13T21:18:20.288ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12362024-12-13T21:18:20.288ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12372024-12-13T21:18:20.288ZDEBGtest_mgs_metrics (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:44649/sled-agents/039be560-54cc-49e3-88df-1a29dadbf913", status: 204, headers: {"x-request-id": "00f90631-d375-41f9-b885-b7f8a5e8d084", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} }) sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12382024-12-13T21:18:20.289ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12392024-12-13T21:18:20.289ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12402024-12-13T21:18:20.289ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12412024-12-13T21:18:20.290ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12422024-12-13T21:18:20.291ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12432024-12-13T21:18:20.291ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12442024-12-13T21:18:20.291ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12452024-12-13T21:18:20.291ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12462024-12-13T21:18:20.291ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12472024-12-13T21:18:20.292ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12482024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12492024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12502024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37757 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12512024-12-13T21:18:20.293ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:37757 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12522024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:37757 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12532024-12-13T21:18:20.293ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:37757 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12542024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12552024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12562024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12572024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12582024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12592024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12602024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12612024-12-13T21:18:20.293ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12622024-12-13T21:18:20.294ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46513 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12632024-12-13T21:18:20.294ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:46513 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12642024-12-13T21:18:20.294ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:46513 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12652024-12-13T21:18:20.295ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:46513 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12662024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12672024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12682024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12692024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12702024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12712024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12722024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12732024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12742024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38381 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12752024-12-13T21:18:20.295ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38381 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12762024-12-13T21:18:20.295ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38381 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12772024-12-13T21:18:20.295ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38381 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12782024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12792024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12802024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12812024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12822024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12832024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12842024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12852024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12862024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43659 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12872024-12-13T21:18:20.297ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43659 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12882024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43659 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12892024-12-13T21:18:20.297ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43659 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12902024-12-13T21:18:20.297ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12912024-12-13T21:18:20.298ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12922024-12-13T21:18:20.298ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12932024-12-13T21:18:20.298ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12942024-12-13T21:18:20.299ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12952024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12962024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12972024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12982024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34851 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12992024-12-13T21:18:20.301ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:34851 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13002024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:34851 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13012024-12-13T21:18:20.301ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:34851 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13022024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13032024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13042024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13052024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13062024-12-13T21:18:20.301ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13072024-12-13T21:18:20.303ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13082024-12-13T21:18:20.304ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13092024-12-13T21:18:20.305ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13102024-12-13T21:18:20.305ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43555 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13112024-12-13T21:18:20.305ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43555 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13122024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43555 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13132024-12-13T21:18:20.306ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43555 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13142024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13152024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13162024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13172024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13182024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13192024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13202024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13212024-12-13T21:18:20.306ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13222024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46643 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13232024-12-13T21:18:20.308ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:46643 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13242024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:46643 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13252024-12-13T21:18:20.308ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:46643 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13262024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13272024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13282024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13292024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13302024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13312024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13322024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13332024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13342024-12-13T21:18:20.308ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40225 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13352024-12-13T21:18:20.308ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:40225 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13362024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:40225 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13372024-12-13T21:18:20.309ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:40225 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13382024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13392024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13402024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13412024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13422024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13432024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13442024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13452024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13462024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37407 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13472024-12-13T21:18:20.309ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:37407 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13482024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:37407 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13492024-12-13T21:18:20.309ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:37407 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13502024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13512024-12-13T21:18:20.309ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13522024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13532024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13542024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13552024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13562024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13572024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13582024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44093 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13592024-12-13T21:18:20.310ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:44093 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13602024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:44093 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13612024-12-13T21:18:20.310ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:44093 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13622024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (init_with_steps): Running step start_crucible_pantry
13632024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = GET path = /crucible/pantry/0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13642024-12-13T21:18:20.310ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = GET path = /crucible/pantry/0/job/{id}/is-finished sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13652024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = GET path = /crucible/pantry/0/job/{id}/ok sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13662024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = DELETE path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13672024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = GET path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13682024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = POST path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13692024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = POST path = /crucible/pantry/0/volume/{id}/background sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13702024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = POST path = /crucible/pantry/0/volume/{id}/bulk-write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13712024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = POST path = /crucible/pantry/0/volume/{id}/import-from-url sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13722024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = POST path = /crucible/pantry/0/volume/{id}/scrub sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13732024-12-13T21:18:20.312ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:33815 method = POST path = /crucible/pantry/0/volume/{id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13742024-12-13T21:18:20.312ZINFOtest_mgs_metrics (dropshot): listening kind = pantry local_addr = [::1]:33815 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13752024-12-13T21:18:20.313ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes kind = pantry local_addr = [::1]:33815 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13762024-12-13T21:18:20.313ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): Started Simulated Crucible Pantry address = [::1]:33815 kind = pantry sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13772024-12-13T21:18:20.315ZDEBGtest_mgs_metrics (init_with_steps): Running step populate_internal_dns
13782024-12-13T21:18:20.315ZDEBGtest_mgs_metrics: Populating Internal DNS
13792024-12-13T21:18:20.346ZINFOtest_mgs_metrics: DNS population: DnsConfigParams { generation: Generation( 1, ), time_created: 2024-12-13T21:18:20.346569336Z, zones: [ DnsConfigZone { zone_name: "control-plane.oxide.internal", records: { "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host": [ Aaaa( ::1, ), ], "34a5b931-88bb-4233-aa25-96630979480d.host": [ Aaaa( ::1, ), ], "_clickhouse._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 42229, target: "f23245e2-d1a5-4886-943a-791e55b53032.host.control-plane.oxide.internal", }, ), ], "_external-dns._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 41531, target: "ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal", }, ), ], "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host": [ Aaaa( ::1, ), ], "_mgs._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 41771, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 39497, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_crucible-pantry._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 33815, target: "e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host.control-plane.oxide.internal", }, ), ], "_mgd._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 44263, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 35889, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_dendrite._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 42289, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 34813, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host": [ Aaaa( ::1, ), ], "97039868-0386-49ec-9259-75c36edb1ac3.host": [ Aaaa( ::ffff:127.0.0.1, ), ], "646b21d1-2847-41b4-aa08-aa491ef3a3f5.host": [ Aaaa( ::1, ), ], "ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host": [ Aaaa( ::1, ), ], "f23245e2-d1a5-4886-943a-791e55b53032.host": [ Aaaa( ::1, ), ], "_clickhouse-native._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "f23245e2-d1a5-4886-943a-791e55b53032.host.control-plane.oxide.internal", }, ), ], "_cockroach._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 45213, target: "646b21d1-2847-41b4-aa08-aa491ef3a3f5.host.control-plane.oxide.internal", }, ), ], "_nameservice._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 39439, target: "34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal", }, ), ], "_nexus._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 44649, target: "97039868-0386-49ec-9259-75c36edb1ac3.host.control-plane.oxide.internal", }, ), ], }, }, ], }
13802024-12-13T21:18:20.347ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:39439/config
13812024-12-13T21:18:20.349ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13822024-12-13T21:18:20.354ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
13832024-12-13T21:18:20.354ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
13842024-12-13T21:18:20.354ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
13852024-12-13T21:18:20.355ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
13862024-12-13T21:18:20.356ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13872024-12-13T21:18:20.361ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13882024-12-13T21:18:20.362ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
13892024-12-13T21:18:20.362ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13902024-12-13T21:18:20.362ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13912024-12-13T21:18:20.366ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13922024-12-13T21:18:20.375ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13932024-12-13T21:18:20.375ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13942024-12-13T21:18:20.375ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13952024-12-13T21:18:20.376ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13962024-12-13T21:18:20.376ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
13972024-12-13T21:18:20.376ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
13982024-12-13T21:18:20.378ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:39439 remote_addr = [::1]:55268
13992024-12-13T21:18:20.385ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14002024-12-13T21:18:20.385ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14012024-12-13T21:18:20.386ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14022024-12-13T21:18:20.389ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14032024-12-13T21:18:20.390ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14042024-12-13T21:18:20.391ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14052024-12-13T21:18:20.391ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a9 2e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53186 req_id = 994a8538-5416-4da3-ae11-5bb03b123519
14062024-12-13T21:18:20.391ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 43310,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53186 req_id = 994a8538-5416-4da3-ae11-5bb03b123519
14072024-12-13T21:18:20.392ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 43310, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53186 req_id = 994a8538-5416-4da3-ae11-5bb03b123519
14082024-12-13T21:18:20.393ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:39439 method = PUT remote_addr = [::1]:55268 req_id = c4447101-d524-4787-957c-607a7fb991e7 uri = /config
14092024-12-13T21:18:20.394ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
14102024-12-13T21:18:20.394ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 108.005487ms iteration = 3 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
14112024-12-13T21:18:20.401ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 1 req_id = c4447101-d524-4787-957c-607a7fb991e7
14122024-12-13T21:18:20.401ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
14132024-12-13T21:18:20.401ZDEBGtest_mgs_metrics (store): creating tree tree_name = generation_1_zone_control-plane.oxide.internal
14142024-12-13T21:18:20.403ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14152024-12-13T21:18:20.403ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14162024-12-13T21:18:20.403ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14172024-12-13T21:18:20.404ZDEBGtest_mgs_metrics (store): updating current config
14182024-12-13T21:18:20.404ZDEBGtest_mgs_metrics (store): flushing default tree
14192024-12-13T21:18:20.406ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
14202024-12-13T21:18:20.406ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14212024-12-13T21:18:20.406ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14222024-12-13T21:18:20.406ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14232024-12-13T21:18:20.407ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14242024-12-13T21:18:20.408ZINFOtest_mgs_metrics (store): pruning trees for generations older than 1 keep = 3
14252024-12-13T21:18:20.408ZINFOtest_mgs_metrics (store): updated generation new_generation = 1 req_id = c4447101-d524-4787-957c-607a7fb991e7
14262024-12-13T21:18:20.408ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 14465 local_addr = [::1]:39439 method = PUT remote_addr = [::1]:55268 req_id = c4447101-d524-4787-957c-607a7fb991e7 response_code = 204 uri = /config
14272024-12-13T21:18:20.409ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:39439/config", status: 204, headers: {"x-request-id": "c4447101-d524-4787-957c-607a7fb991e7", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
14282024-12-13T21:18:20.410ZDEBGtest_mgs_metrics (init_with_steps): Running step configure_sled_agent1
14292024-12-13T21:18:20.444ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:45365/omicron-zones
14302024-12-13T21:18:20.446ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14312024-12-13T21:18:20.447ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14322024-12-13T21:18:20.447ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14332024-12-13T21:18:20.447ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14342024-12-13T21:18:20.447ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45365 remote_addr = [::1]:41852 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14352024-12-13T21:18:20.452ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14362024-12-13T21:18:20.452ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14372024-12-13T21:18:20.452ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14382024-12-13T21:18:20.453ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14392024-12-13T21:18:20.454ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41852 req_id = f1de502a-604a-4fa9-abb0-c8d6335985c7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-zones
14402024-12-13T21:18:20.470ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 17070 local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41852 req_id = f1de502a-604a-4fa9-abb0-c8d6335985c7 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-zones
14412024-12-13T21:18:20.471ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:45365/omicron-zones", status: 204, headers: {"x-request-id": "f1de502a-604a-4fa9-abb0-c8d6335985c7", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
14422024-12-13T21:18:20.472ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:45365/network-bootstore-config
14432024-12-13T21:18:20.474ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41852 req_id = 1b509650-f2ef-4ba0-a92e-5a253e9528a5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
14442024-12-13T21:18:20.475ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 498 local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41852 req_id = 1b509650-f2ef-4ba0-a92e-5a253e9528a5 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
14452024-12-13T21:18:20.477ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:45365/network-bootstore-config", status: 204, headers: {"x-request-id": "1b509650-f2ef-4ba0-a92e-5a253e9528a5", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
14462024-12-13T21:18:20.478ZDEBGtest_mgs_metrics (init_with_steps): Running step configure_sled_agent2
14472024-12-13T21:18:20.566ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:42715/omicron-zones
14482024-12-13T21:18:20.567ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14492024-12-13T21:18:20.575ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
14502024-12-13T21:18:20.575ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
14512024-12-13T21:18:20.575ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14522024-12-13T21:18:20.575ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14532024-12-13T21:18:20.576ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14542024-12-13T21:18:20.577ZINFOtest_mgs_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
14552024-12-13T21:18:20.578ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:45365/eip-gateways
14562024-12-13T21:18:20.696ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager body = None method = GET uri = http://[::1]:42715/v2p
14572024-12-13T21:18:20.699ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14582024-12-13T21:18:20.699ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14592024-12-13T21:18:20.699ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14602024-12-13T21:18:20.700ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:42715 remote_addr = [::1]:41960 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
14612024-12-13T21:18:20.701ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:42715 remote_addr = [::1]:41962 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
14622024-12-13T21:18:20.701ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41848 req_id = afbf2a92-deef-45a0-9b72-6d18684b4363 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
14632024-12-13T21:18:20.703ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 2780 local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41848 req_id = afbf2a92-deef-45a0-9b72-6d18684b4363 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
14642024-12-13T21:18:20.704ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:42715 method = GET remote_addr = [::1]:41962 req_id = bc525095-8718-41a6-9896-f7b989501a4d sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
14652024-12-13T21:18:20.705ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:45365/eip-gateways", status: 204, headers: {"x-request-id": "afbf2a92-deef-45a0-9b72-6d18684b4363", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
14662024-12-13T21:18:20.705ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 2.066855353s iteration = 1 status = {}
14672024-12-13T21:18:20.705ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 1562 local_addr = [::1]:42715 method = GET remote_addr = [::1]:41962 req_id = bc525095-8718-41a6-9896-f7b989501a4d response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
14682024-12-13T21:18:20.706ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:42715 method = PUT remote_addr = [::1]:41960 req_id = dd4566f0-b470-4953-b4e2-80703212cb85 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /omicron-zones
14692024-12-13T21:18:20.707ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:42715/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc525095-8718-41a6-9896-f7b989501a4d", "content-length": "2", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
14702024-12-13T21:18:20.708ZINFOtest_mgs_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-039be560
14712024-12-13T21:18:20.708ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-039be560
14722024-12-13T21:18:20.708ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-039be560
14732024-12-13T21:18:20.709ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:45365/v2p
14742024-12-13T21:18:20.710ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 3649 local_addr = [::1]:42715 method = PUT remote_addr = [::1]:41960 req_id = dd4566f0-b470-4953-b4e2-80703212cb85 response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /omicron-zones
14752024-12-13T21:18:20.710ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14762024-12-13T21:18:20.710ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14772024-12-13T21:18:20.711ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14782024-12-13T21:18:20.712ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45365 remote_addr = [::1]:41854 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14792024-12-13T21:18:20.713ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:42715/omicron-zones", status: 204, headers: {"x-request-id": "dd4566f0-b470-4953-b4e2-80703212cb85", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
14802024-12-13T21:18:20.713ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:42715/network-bootstore-config
14812024-12-13T21:18:20.713ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45365 method = GET remote_addr = [::1]:41854 req_id = 2ae6edd9-7a6a-473e-b288-9d740e82eb50 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
14822024-12-13T21:18:20.714ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 229 local_addr = [::1]:45365 method = GET remote_addr = [::1]:41854 req_id = 2ae6edd9-7a6a-473e-b288-9d740e82eb50 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
14832024-12-13T21:18:20.714ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:42715 method = PUT remote_addr = [::1]:41960 req_id = 8e69f53f-9edb-405c-a440-1580647c5119 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /network-bootstore-config
14842024-12-13T21:18:20.714ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:45365/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ae6edd9-7a6a-473e-b288-9d740e82eb50", "content-length": "2", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
14852024-12-13T21:18:20.715ZINFOtest_mgs_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
14862024-12-13T21:18:20.715ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
14872024-12-13T21:18:20.715ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
14882024-12-13T21:18:20.715ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 2.085921407s iteration = 1 status = {}
14892024-12-13T21:18:20.715ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 1276 local_addr = [::1]:42715 method = PUT remote_addr = [::1]:41960 req_id = 8e69f53f-9edb-405c-a440-1580647c5119 response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /network-bootstore-config
14902024-12-13T21:18:20.715ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14912024-12-13T21:18:20.715ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14922024-12-13T21:18:20.716ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14932024-12-13T21:18:20.717ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:42715/network-bootstore-config", status: 204, headers: {"x-request-id": "8e69f53f-9edb-405c-a440-1580647c5119", "date": "Fri, 13 Dec 2024 21:18:19 GMT"} })
14942024-12-13T21:18:20.718ZDEBGtest_mgs_metrics (init_with_steps): Running step start_nexus_external
14952024-12-13T21:18:20.718ZDEBGtest_mgs_metrics: Starting Nexus (external API)
14962024-12-13T21:18:21.641ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
14972024-12-13T21:18:21.641ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14982024-12-13T21:18:21.641ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14992024-12-13T21:18:21.641ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15002024-12-13T21:18:21.642ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15012024-12-13T21:18:21.645ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
15022024-12-13T21:18:21.645ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15032024-12-13T21:18:21.645ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15042024-12-13T21:18:21.645ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15052024-12-13T21:18:21.646ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15062024-12-13T21:18:21.647ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15072024-12-13T21:18:21.650ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15082024-12-13T21:18:21.650ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15092024-12-13T21:18:21.650ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15102024-12-13T21:18:21.651ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15112024-12-13T21:18:21.655ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15122024-12-13T21:18:21.655ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15132024-12-13T21:18:21.655ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15142024-12-13T21:18:21.656ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15152024-12-13T21:18:21.665ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15162024-12-13T21:18:21.665ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15172024-12-13T21:18:21.665ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15182024-12-13T21:18:21.667ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15192024-12-13T21:18:21.671ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15202024-12-13T21:18:21.671ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15212024-12-13T21:18:21.671ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15222024-12-13T21:18:21.680ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15232024-12-13T21:18:21.689ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15242024-12-13T21:18:21.689ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
15252024-12-13T21:18:21.689ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
15262024-12-13T21:18:21.689ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15272024-12-13T21:18:21.689ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15282024-12-13T21:18:21.689ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15292024-12-13T21:18:21.690ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15302024-12-13T21:18:21.697ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15312024-12-13T21:18:21.698ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15322024-12-13T21:18:21.698ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15332024-12-13T21:18:21.698ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15342024-12-13T21:18:21.709ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15352024-12-13T21:18:21.710ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15362024-12-13T21:18:21.710ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15372024-12-13T21:18:21.710ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15382024-12-13T21:18:21.710ZINFOtest_mgs_metrics (InternalApi): set cockroachdb setting setting = cluster.preserve_downgrade_option value = 22.1
15392024-12-13T21:18:21.710ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15402024-12-13T21:18:21.710ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15412024-12-13T21:18:21.711ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15422024-12-13T21:18:21.715ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15432024-12-13T21:18:21.725ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15442024-12-13T21:18:21.725ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15452024-12-13T21:18:21.725ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15462024-12-13T21:18:21.726ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15472024-12-13T21:18:21.732ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
15482024-12-13T21:18:21.733ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15492024-12-13T21:18:21.733ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15502024-12-13T21:18:21.733ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15512024-12-13T21:18:21.733ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15522024-12-13T21:18:21.740ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15532024-12-13T21:18:21.749ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
15542024-12-13T21:18:21.749ZDEBGtest_mgs_metrics (InternalApi): updating rack subnet for rack c19a698f-c6f9-4a17-ae30-20d711b8f7dc to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), )
15552024-12-13T21:18:21.749ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15562024-12-13T21:18:21.749ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15572024-12-13T21:18:21.749ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15582024-12-13T21:18:21.758ZINFOtest_mgs_metrics (InternalApi): Using static configuration for external switchports
15592024-12-13T21:18:21.759ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15602024-12-13T21:18:21.759ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15612024-12-13T21:18:21.759ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15622024-12-13T21:18:21.773ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15632024-12-13T21:18:21.775ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15642024-12-13T21:18:21.775ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15652024-12-13T21:18:21.778ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15662024-12-13T21:18:21.780ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15672024-12-13T21:18:21.780ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15682024-12-13T21:18:21.781ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15692024-12-13T21:18:21.783ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15702024-12-13T21:18:21.783ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15712024-12-13T21:18:21.786ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15722024-12-13T21:18:21.788ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15732024-12-13T21:18:21.788ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15742024-12-13T21:18:21.796ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
15752024-12-13T21:18:21.797ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15762024-12-13T21:18:21.798ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15772024-12-13T21:18:21.798ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15782024-12-13T21:18:21.801ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15792024-12-13T21:18:21.804ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15802024-12-13T21:18:21.805ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15812024-12-13T21:18:21.805ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15822024-12-13T21:18:21.805ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15832024-12-13T21:18:21.808ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15842024-12-13T21:18:21.808ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15852024-12-13T21:18:21.808ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15862024-12-13T21:18:21.809ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15872024-12-13T21:18:21.811ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15882024-12-13T21:18:21.820ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
15892024-12-13T21:18:21.823ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
15902024-12-13T21:18:21.823ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
15912024-12-13T21:18:21.823ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15922024-12-13T21:18:21.823ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15932024-12-13T21:18:21.823ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15942024-12-13T21:18:21.823ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15952024-12-13T21:18:21.824ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15962024-12-13T21:18:21.824ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15972024-12-13T21:18:21.824ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15982024-12-13T21:18:21.825ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15992024-12-13T21:18:21.827ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16002024-12-13T21:18:21.827ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16012024-12-13T21:18:21.827ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16022024-12-13T21:18:21.827ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16032024-12-13T21:18:21.831ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16042024-12-13T21:18:21.844ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16052024-12-13T21:18:21.847ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16062024-12-13T21:18:21.847ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16072024-12-13T21:18:21.847ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16082024-12-13T21:18:21.853ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
16092024-12-13T21:18:21.853ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet
16102024-12-13T21:18:21.854ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16112024-12-13T21:18:21.854ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16122024-12-13T21:18:21.854ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16132024-12-13T21:18:21.870ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16142024-12-13T21:18:21.872ZINFOtest_mgs_metrics (InternalApi): Recording Rack Network Configuration
16152024-12-13T21:18:21.872ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16162024-12-13T21:18:21.872ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16172024-12-13T21:18:21.873ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16182024-12-13T21:18:21.873ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16192024-12-13T21:18:21.878ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
16202024-12-13T21:18:21.878ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16212024-12-13T21:18:21.878ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16222024-12-13T21:18:21.879ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16232024-12-13T21:18:21.888ZTRACtest_mgs_metrics (service-firewall-rule-progator): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-12-13T20:56:30.038685Z, time_modified: 2024-12-13T20:56:30.038685Z, allowed_ips: None } background_task = service_firewall_rule_propagation
16242024-12-13T21:18:21.888ZDEBGtest_mgs_metrics (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
16252024-12-13T21:18:21.888ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
16262024-12-13T21:18:21.888ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
16272024-12-13T21:18:22.105ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16282024-12-13T21:18:22.105ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16292024-12-13T21:18:22.105ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16302024-12-13T21:18:22.105ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16312024-12-13T21:18:22.105ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16322024-12-13T21:18:22.119ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16332024-12-13T21:18:22.123ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16342024-12-13T21:18:22.123ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16352024-12-13T21:18:22.124ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16362024-12-13T21:18:22.125ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16372024-12-13T21:18:22.149ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 943d1a53-9789-4bf9-9ed7-aee3fd1bc2e0, lookup_type: ByName("oxide-service-pool") }
16382024-12-13T21:18:22.149ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16392024-12-13T21:18:22.149ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16402024-12-13T21:18:22.149ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16412024-12-13T21:18:22.150ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16422024-12-13T21:18:22.179ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16432024-12-13T21:18:22.212ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 943d1a53-9789-4bf9-9ed7-aee3fd1bc2e0, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16442024-12-13T21:18:22.217ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16452024-12-13T21:18:22.218ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16462024-12-13T21:18:22.218ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16472024-12-13T21:18:22.254ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
16482024-12-13T21:18:22.254ZDEBGtest_mgs_metrics (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
16492024-12-13T21:18:22.254ZINFOtest_mgs_metrics (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation
16502024-12-13T21:18:22.254ZDEBGtest_mgs_metrics (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 3.616390904s
16512024-12-13T21:18:22.254ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 3.61664957s iteration = 1 status = {}
16522024-12-13T21:18:22.261ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 943d1a53-9789-4bf9-9ed7-aee3fd1bc2e0, lookup_type: ByName("oxide-service-pool") }
16532024-12-13T21:18:22.261ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16542024-12-13T21:18:22.261ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16552024-12-13T21:18:22.261ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16562024-12-13T21:18:22.271ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16572024-12-13T21:18:22.280ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16582024-12-13T21:18:22.288ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 943d1a53-9789-4bf9-9ed7-aee3fd1bc2e0, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16592024-12-13T21:18:22.326ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 943d1a53-9789-4bf9-9ed7-aee3fd1bc2e0, lookup_type: ByName("oxide-service-pool") }
16602024-12-13T21:18:22.327ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16612024-12-13T21:18:22.327ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16622024-12-13T21:18:22.327ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16632024-12-13T21:18:22.327ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16642024-12-13T21:18:22.333ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16652024-12-13T21:18:22.344ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 943d1a53-9789-4bf9-9ed7-aee3fd1bc2e0, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16662024-12-13T21:18:22.347ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
16672024-12-13T21:18:22.348ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16682024-12-13T21:18:22.349ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16692024-12-13T21:18:22.349ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16702024-12-13T21:18:22.349ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16712024-12-13T21:18:22.352ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16722024-12-13T21:18:22.356ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
16732024-12-13T21:18:22.718ZINFOtest_mgs_metrics (InternalApi): inserted blueprint blueprint_id = 3aa4941d-8030-4b87-b040-fc64e1526a3a
16742024-12-13T21:18:22.718ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
16752024-12-13T21:18:22.718ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16762024-12-13T21:18:22.718ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16772024-12-13T21:18:22.718ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16782024-12-13T21:18:22.718ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16792024-12-13T21:18:22.723ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16802024-12-13T21:18:22.724ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
16812024-12-13T21:18:22.730ZINFOtest_mgs_metrics (InternalApi): No networking records needed for cockroach_db service
16822024-12-13T21:18:22.730ZINFOtest_mgs_metrics (InternalApi): No networking records needed for clickhouse service
16832024-12-13T21:18:22.730ZINFOtest_mgs_metrics (InternalApi): No networking records needed for internal_dns service
16842024-12-13T21:18:22.926ZINFOtest_mgs_metrics (InternalApi): Inserted networking records for external_dns service
16852024-12-13T21:18:22.985ZINFOtest_mgs_metrics (InternalApi): Inserted networking records for nexus service
16862024-12-13T21:18:22.985ZINFOtest_mgs_metrics (InternalApi): No networking records needed for crucible_pantry service
16872024-12-13T21:18:22.985ZINFOtest_mgs_metrics (InternalApi): Inserted service networking records
16882024-12-13T21:18:22.985ZINFOtest_mgs_metrics (InternalApi): Inserted physical disks
16892024-12-13T21:18:22.985ZINFOtest_mgs_metrics (InternalApi): Inserted zpools
16902024-12-13T21:18:22.985ZINFOtest_mgs_metrics (InternalApi): Inserted datasets
16912024-12-13T21:18:23.010ZINFOtest_mgs_metrics (InternalApi): Populated DNS tables for internal DNS
16922024-12-13T21:18:23.021ZINFOtest_mgs_metrics (InternalApi): Populated DNS tables for external DNS
16932024-12-13T21:18:23.022ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16942024-12-13T21:18:23.023ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16952024-12-13T21:18:23.024ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16962024-12-13T21:18:23.024ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16972024-12-13T21:18:23.026ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16982024-12-13T21:18:23.039ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16992024-12-13T21:18:23.048ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17002024-12-13T21:18:23.052ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
17012024-12-13T21:18:23.052ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17022024-12-13T21:18:23.052ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17032024-12-13T21:18:23.053ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17042024-12-13T21:18:23.053ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17052024-12-13T21:18:23.064ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17062024-12-13T21:18:23.068ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17072024-12-13T21:18:23.096ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
17082024-12-13T21:18:23.096ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17092024-12-13T21:18:23.097ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17102024-12-13T21:18:23.097ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17112024-12-13T21:18:23.097ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17122024-12-13T21:18:23.100ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17132024-12-13T21:18:23.124ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
17142024-12-13T21:18:23.126ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
17152024-12-13T21:18:23.126ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17162024-12-13T21:18:23.126ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17172024-12-13T21:18:23.126ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17182024-12-13T21:18:23.127ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17192024-12-13T21:18:23.141ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17202024-12-13T21:18:23.148ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
17212024-12-13T21:18:23.151ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
17222024-12-13T21:18:23.151ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17232024-12-13T21:18:23.151ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17242024-12-13T21:18:23.151ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17252024-12-13T21:18:23.152ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17262024-12-13T21:18:23.158ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17272024-12-13T21:18:23.178ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
17282024-12-13T21:18:23.194ZINFOtest_mgs_metrics (InternalApi): Created recovery silo
17292024-12-13T21:18:23.196ZINFOtest_mgs_metrics (InternalApi): Created recovery user
17302024-12-13T21:18:23.218ZINFOtest_mgs_metrics (InternalApi): Created recovery user's password
17312024-12-13T21:18:23.221ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 00f97f81-909e-4b96-81e4-53ffb6a12150, lookup_type: ById(00f97f81-909e-4b96-81e4-53ffb6a12150) }
17322024-12-13T21:18:23.221ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 00f97f81-909e-4b96-81e4-53ffb6a12150 resource_type = Silo
17332024-12-13T21:18:23.221ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17342024-12-13T21:18:23.221ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17352024-12-13T21:18:23.221ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17362024-12-13T21:18:23.248ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17372024-12-13T21:18:23.249ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17382024-12-13T21:18:23.251ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17392024-12-13T21:18:23.251ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17402024-12-13T21:18:23.259ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17412024-12-13T21:18:23.267ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 00f97f81-909e-4b96-81e4-53ffb6a12150, lookup_type: ById(00f97f81-909e-4b96-81e4-53ffb6a12150) } result = Ok(())
17422024-12-13T21:18:23.267ZDEBGtest_mgs_metrics (InternalApi): Generated role assignment queries
17432024-12-13T21:18:23.270ZINFOtest_mgs_metrics (InternalApi): Granted Silo privileges
17442024-12-13T21:18:23.272ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
17452024-12-13T21:18:23.272ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17462024-12-13T21:18:23.272ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17472024-12-13T21:18:23.272ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17482024-12-13T21:18:23.272ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17492024-12-13T21:18:23.287ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17502024-12-13T21:18:23.290ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17512024-12-13T21:18:23.308ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17522024-12-13T21:18:23.309ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17532024-12-13T21:18:23.309ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17542024-12-13T21:18:23.314ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17552024-12-13T21:18:23.315ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17562024-12-13T21:18:23.315ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17572024-12-13T21:18:23.321ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17582024-12-13T21:18:23.322ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17592024-12-13T21:18:23.325ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17602024-12-13T21:18:23.334ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17612024-12-13T21:18:23.334ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17622024-12-13T21:18:23.334ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17632024-12-13T21:18:23.334ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17642024-12-13T21:18:23.336ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17652024-12-13T21:18:23.340ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17662024-12-13T21:18:23.340ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17672024-12-13T21:18:23.340ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17682024-12-13T21:18:23.343ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17692024-12-13T21:18:23.345ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17702024-12-13T21:18:23.345ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17712024-12-13T21:18:23.345ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17722024-12-13T21:18:23.346ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17732024-12-13T21:18:23.349ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17742024-12-13T21:18:23.357ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17752024-12-13T21:18:23.357ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
17762024-12-13T21:18:23.357ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17772024-12-13T21:18:23.357ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17782024-12-13T21:18:23.357ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17792024-12-13T21:18:23.360ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17802024-12-13T21:18:23.360ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17812024-12-13T21:18:23.361ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17822024-12-13T21:18:23.361ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17832024-12-13T21:18:23.361ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17842024-12-13T21:18:23.361ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17852024-12-13T21:18:23.373ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17862024-12-13T21:18:23.373ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17872024-12-13T21:18:23.373ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17882024-12-13T21:18:23.375ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17892024-12-13T21:18:23.382ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17902024-12-13T21:18:23.406ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17912024-12-13T21:18:23.407ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17922024-12-13T21:18:23.407ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17932024-12-13T21:18:23.407ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17942024-12-13T21:18:23.412ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17952024-12-13T21:18:23.414ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17962024-12-13T21:18:23.415ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17972024-12-13T21:18:23.421ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17982024-12-13T21:18:23.421ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17992024-12-13T21:18:23.423ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18002024-12-13T21:18:23.440ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18012024-12-13T21:18:23.440ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18022024-12-13T21:18:23.441ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18032024-12-13T21:18:23.446ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
18042024-12-13T21:18:23.450ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18052024-12-13T21:18:23.450ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18062024-12-13T21:18:23.450ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18072024-12-13T21:18:23.450ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18082024-12-13T21:18:23.455ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18092024-12-13T21:18:23.455ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18102024-12-13T21:18:23.455ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18112024-12-13T21:18:23.466ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18122024-12-13T21:18:23.474ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18132024-12-13T21:18:23.475ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18142024-12-13T21:18:23.475ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18152024-12-13T21:18:23.476ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18162024-12-13T21:18:23.484ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18172024-12-13T21:18:23.506ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
18182024-12-13T21:18:23.507ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
18192024-12-13T21:18:23.507ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18202024-12-13T21:18:23.507ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18212024-12-13T21:18:23.508ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18222024-12-13T21:18:23.514ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18232024-12-13T21:18:23.514ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18242024-12-13T21:18:23.515ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18252024-12-13T21:18:23.519ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18262024-12-13T21:18:23.520ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18272024-12-13T21:18:23.521ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18282024-12-13T21:18:23.529ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18292024-12-13T21:18:23.529ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18302024-12-13T21:18:23.530ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18312024-12-13T21:18:23.532ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
18322024-12-13T21:18:23.532ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18332024-12-13T21:18:23.533ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18342024-12-13T21:18:23.533ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18352024-12-13T21:18:23.534ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18362024-12-13T21:18:23.536ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18372024-12-13T21:18:23.536ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18382024-12-13T21:18:23.536ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18392024-12-13T21:18:23.537ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18402024-12-13T21:18:23.539ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18412024-12-13T21:18:23.539ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18422024-12-13T21:18:23.539ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18432024-12-13T21:18:23.541ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18442024-12-13T21:18:23.542ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18452024-12-13T21:18:23.566ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
18462024-12-13T21:18:23.566ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
18472024-12-13T21:18:23.566ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
18482024-12-13T21:18:23.566ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18492024-12-13T21:18:23.566ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18502024-12-13T21:18:23.566ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18512024-12-13T21:18:23.571ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18522024-12-13T21:18:23.596ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18532024-12-13T21:18:23.597ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18542024-12-13T21:18:23.597ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18552024-12-13T21:18:23.598ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18562024-12-13T21:18:23.601ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18572024-12-13T21:18:23.601ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18582024-12-13T21:18:23.601ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18592024-12-13T21:18:23.601ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18602024-12-13T21:18:23.610ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18612024-12-13T21:18:23.614ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
18622024-12-13T21:18:23.614ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18632024-12-13T21:18:23.616ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18642024-12-13T21:18:23.616ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18652024-12-13T21:18:23.639ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18662024-12-13T21:18:23.639ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18672024-12-13T21:18:23.639ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18682024-12-13T21:18:23.642ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18692024-12-13T21:18:23.645ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18702024-12-13T21:18:23.645ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18712024-12-13T21:18:23.647ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18722024-12-13T21:18:23.647ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18732024-12-13T21:18:23.648ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18742024-12-13T21:18:23.657ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18752024-12-13T21:18:23.657ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18762024-12-13T21:18:23.657ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18772024-12-13T21:18:23.658ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
18782024-12-13T21:18:23.659ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18792024-12-13T21:18:23.660ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18802024-12-13T21:18:23.660ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18812024-12-13T21:18:23.660ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18822024-12-13T21:18:23.664ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18832024-12-13T21:18:23.664ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18842024-12-13T21:18:23.664ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18852024-12-13T21:18:23.665ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18862024-12-13T21:18:23.673ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18872024-12-13T21:18:23.673ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18882024-12-13T21:18:23.673ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18892024-12-13T21:18:23.674ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18902024-12-13T21:18:23.676ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18912024-12-13T21:18:23.698ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18922024-12-13T21:18:23.698ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
18932024-12-13T21:18:23.698ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
18942024-12-13T21:18:23.698ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18952024-12-13T21:18:23.698ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18962024-12-13T21:18:23.698ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18972024-12-13T21:18:23.699ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18982024-12-13T21:18:23.708ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18992024-12-13T21:18:23.709ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19002024-12-13T21:18:23.709ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19012024-12-13T21:18:23.710ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19022024-12-13T21:18:23.712ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19032024-12-13T21:18:23.712ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19042024-12-13T21:18:23.713ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19052024-12-13T21:18:23.714ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19062024-12-13T21:18:23.718ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19072024-12-13T21:18:23.732ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
19082024-12-13T21:18:23.738ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19092024-12-13T21:18:23.739ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19102024-12-13T21:18:23.740ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19112024-12-13T21:18:23.748ZDEBGtest_mgs_metrics (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 184b8d5c-4a61-45dd-a4eb-4a4a6b73719b, kind: Service { id: 97039868-0386-49ec-9259-75c36edb1ac3 }, name: Name("nexus-97039868-0386-49ec-9259-75c36edb1ac3"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 07b099d9-af6c-4a52-8be7-26d4b3e67527, kind: Service { id: ab3c6976-382d-4564-bfcf-cb6a75ba0c83 }, name: Name("external-dns-ab3c6976-382d-4564-bfcf-cb6a75ba0c83"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
19122024-12-13T21:18:23.748ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
19132024-12-13T21:18:23.749ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19142024-12-13T21:18:23.749ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19152024-12-13T21:18:23.749ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19162024-12-13T21:18:23.749ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19172024-12-13T21:18:23.754ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19182024-12-13T21:18:23.763ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
19192024-12-13T21:18:23.763ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19202024-12-13T21:18:23.763ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19212024-12-13T21:18:23.764ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19222024-12-13T21:18:23.768ZTRACtest_mgs_metrics (nexus): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-12-13T20:56:30.038685Z, time_modified: 2024-12-13T21:18:23.291139Z, allowed_ips: None }
19232024-12-13T21:18:23.769ZDEBGtest_mgs_metrics (nexus): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
19242024-12-13T21:18:23.769ZDEBGtest_mgs_metrics (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 07b099d9-af6c-4a52-8be7-26d4b3e67527, kind: Service { id: ab3c6976-382d-4564-bfcf-cb6a75ba0c83 }, name: Name("external-dns-ab3c6976-382d-4564-bfcf-cb6a75ba0c83"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 184b8d5c-4a61-45dd-a4eb-4a4a6b73719b, kind: Service { id: 97039868-0386-49ec-9259-75c36edb1ac3 }, name: Name("nexus-97039868-0386-49ec-9259-75c36edb1ac3"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
19252024-12-13T21:18:23.769ZDEBGtest_mgs_metrics (nexus): resolved 2 rules for sleds
19262024-12-13T21:18:23.810ZDEBGtest_mgs_metrics (nexus): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2024-12-13T21:18:18.665137Z, time_modified: 2024-12-13T21:18:18.665137Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(45365), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
19272024-12-13T21:18:23.811ZDEBGtest_mgs_metrics (nexus): sending firewall rules to sled agents
19282024-12-13T21:18:23.811ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
19292024-12-13T21:18:23.811ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {} }
19302024-12-13T21:18:23.811ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
19312024-12-13T21:18:23.816ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
19322024-12-13T21:18:23.817ZTRACtest_mgs_metrics (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19332024-12-13T21:18:23.817ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
19342024-12-13T21:18:23.817ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {} }
19352024-12-13T21:18:23.817ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
19362024-12-13T21:18:23.820ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
19372024-12-13T21:18:23.826ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
19382024-12-13T21:18:23.897ZDEBGtest_mgs_metrics (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:45365/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19392024-12-13T21:18:23.898ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45365 remote_addr = [::1]:41856 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
19402024-12-13T21:18:23.899ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41856 req_id = f32a84ff-af11-4bbe-8cb6-5ad60bd1af53 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19412024-12-13T21:18:23.900ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 759 local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41856 req_id = f32a84ff-af11-4bbe-8cb6-5ad60bd1af53 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19422024-12-13T21:18:23.900ZDEBGtest_mgs_metrics (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:45365/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "f32a84ff-af11-4bbe-8cb6-5ad60bd1af53", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
19432024-12-13T21:18:23.902ZINFOtest_mgs_metrics (nexus): updated firewall rules on 1 sleds
19442024-12-13T21:18:23.904ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
19452024-12-13T21:18:23.904ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {} }
19462024-12-13T21:18:23.905ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
19472024-12-13T21:18:23.905ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
19482024-12-13T21:18:23.906ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19492024-12-13T21:18:23.906ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19502024-12-13T21:18:23.906ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19512024-12-13T21:18:23.906ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19522024-12-13T21:18:23.906ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
19532024-12-13T21:18:23.906ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
19542024-12-13T21:18:23.907ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
19552024-12-13T21:18:23.907ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19562024-12-13T21:18:23.907ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19572024-12-13T21:18:23.907ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19582024-12-13T21:18:23.907ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19592024-12-13T21:18:23.907ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
19602024-12-13T21:18:23.909ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
19612024-12-13T21:18:23.909ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
19622024-12-13T21:18:23.909ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
19632024-12-13T21:18:23.909ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19642024-12-13T21:18:23.909ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19652024-12-13T21:18:23.909ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19662024-12-13T21:18:23.911ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19672024-12-13T21:18:23.911ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Signaled
19682024-12-13T21:18:23.911ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
19692024-12-13T21:18:23.911ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19702024-12-13T21:18:23.911ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
19712024-12-13T21:18:23.912ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
19722024-12-13T21:18:23.912ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
19732024-12-13T21:18:23.917ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19742024-12-13T21:18:23.917ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19752024-12-13T21:18:23.917ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19762024-12-13T21:18:23.918ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19772024-12-13T21:18:23.918ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19782024-12-13T21:18:23.920ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: 3c 09 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 <............_na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:55212 req_id = a71afc54-e7e4-4ebc-8a3c-de786cb5f615
19792024-12-13T21:18:23.920ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15369,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55212 req_id = a71afc54-e7e4-4ebc-8a3c-de786cb5f615
19802024-12-13T21:18:23.920ZDEBGtest_mgs_metrics (store): query key key = _nameservice._tcp
19812024-12-13T21:18:23.920ZDEBGtest_mgs_metrics (store): query key key = 34a5b931-88bb-4233-aa25-96630979480d.host
19822024-12-13T21:18:23.920ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55212 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39439, target: Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal") })) }] req_id = a71afc54-e7e4-4ebc-8a3c-de786cb5f615
19832024-12-13T21:18:23.920ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: ed c7 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 ............._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51308 req_id = af81cf86-db04-4021-b990-fed0b87863c8
19842024-12-13T21:18:23.920ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60871,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51308 req_id = af81cf86-db04-4021-b990-fed0b87863c8
19852024-12-13T21:18:23.920ZDEBGtest_mgs_metrics (store): query key key = _external-dns._tcp
19862024-12-13T21:18:23.920ZDEBGtest_mgs_metrics (store): query key key = ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host
19872024-12-13T21:18:23.920ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51308 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41531, target: Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal") })) }] req_id = af81cf86-db04-4021-b990-fed0b87863c8
19882024-12-13T21:18:23.921ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19892024-12-13T21:18:23.921ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19902024-12-13T21:18:23.921ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19912024-12-13T21:18:23.921ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19922024-12-13T21:18:23.921ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19932024-12-13T21:18:23.924ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
19942024-12-13T21:18:23.924ZTRACtest_mgs_metrics (ServiceBalancer): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19952024-12-13T21:18:23.924ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
19962024-12-13T21:18:23.924ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {} }
19972024-12-13T21:18:23.924ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
19982024-12-13T21:18:23.924ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39439, target: Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2098, tv_nsec: 706358388 } })
19992024-12-13T21:18:23.924ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
20002024-12-13T21:18:23.924ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41531, target: Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2098, tv_nsec: 706730366 } })
20012024-12-13T21:18:23.924ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
20022024-12-13T21:18:23.928ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20032024-12-13T21:18:23.930ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20042024-12-13T21:18:23.930ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20052024-12-13T21:18:23.930ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20062024-12-13T21:18:23.931ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20072024-12-13T21:18:23.931ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 29 84 01 00 00 01 00 00 00 00 00 01 24 33 34 61 )...........$34a\n0010: 35 62 39 33 31 2d 38 38 62 62 2d 34 32 33 33 2d 5b931-88bb-4233-\n0020: 61 61 32 35 2d 39 36 36 33 30 39 37 39 34 38 30 aa25-96630979480\n0030: 64 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 d.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:64404 req_id = 3420b2e2-a20e-417f-aefe-d1892e6e9e9c
20082024-12-13T21:18:23.932ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10628,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 51,\n 52,\n 97,\n 53,\n 98,\n 57,\n 51,\n 49,\n 45,\n 56,\n 56,\n 98,\n 98,\n 45,\n 52,\n 50,\n 51,\n 51,\n 45,\n 97,\n 97,\n 50,\n 53,\n 45,\n 57,\n 54,\n 54,\n 51,\n 48,\n 57,\n 55,\n 57,\n 52,\n 56,\n 48,\n 100,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64404 req_id = 3420b2e2-a20e-417f-aefe-d1892e6e9e9c
20092024-12-13T21:18:23.932ZDEBGtest_mgs_metrics (store): query key key = 34a5b931-88bb-4233-aa25-96630979480d.host
20102024-12-13T21:18:23.932ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64404 query = LowerQuery { name: LowerName(Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal.")), original: Query { name: Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("34a5b931-88bb-4233-aa25-96630979480d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3420b2e2-a20e-417f-aefe-d1892e6e9e9c
20112024-12-13T21:18:23.932ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 52 1e 01 00 00 01 00 00 00 00 00 01 24 61 62 33 R...........$ab3\n0010: 63 36 39 37 36 2d 33 38 32 64 2d 34 35 36 34 2d c6976-382d-4564-\n0020: 62 66 63 66 2d 63 62 36 61 37 35 62 61 30 63 38 bfcf-cb6a75ba0c8\n0030: 33 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 3.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:49402 req_id = 5d17b0e2-b671-48ce-aca1-5e937186cfa3
20122024-12-13T21:18:23.932ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 21022,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 97,\n 98,\n 51,\n 99,\n 54,\n 57,\n 55,\n 54,\n 45,\n 51,\n 56,\n 50,\n 100,\n 45,\n 52,\n 53,\n 54,\n 52,\n 45,\n 98,\n 102,\n 99,\n 102,\n 45,\n 99,\n 98,\n 54,\n 97,\n 55,\n 53,\n 98,\n 97,\n 48,\n 99,\n 56,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49402 req_id = 5d17b0e2-b671-48ce-aca1-5e937186cfa3
20132024-12-13T21:18:23.932ZDEBGtest_mgs_metrics (store): query key key = ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host
20142024-12-13T21:18:23.932ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49402 query = LowerQuery { name: LowerName(Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal.")), original: Query { name: Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("ab3c6976-382d-4564-bfcf-cb6a75ba0c83.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 5d17b0e2-b671-48ce-aca1-5e937186cfa3
20152024-12-13T21:18:23.935ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20162024-12-13T21:18:23.944ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20172024-12-13T21:18:23.947ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20182024-12-13T21:18:23.964ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20192024-12-13T21:18:23.966ZDEBGtest_mgs_metrics (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
20202024-12-13T21:18:23.966ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
20212024-12-13T21:18:23.966ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
20222024-12-13T21:18:23.966ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
20232024-12-13T21:18:23.966ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20242024-12-13T21:18:23.966ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20252024-12-13T21:18:23.966ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20262024-12-13T21:18:23.966ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20272024-12-13T21:18:23.966ZINFOtest_mgs_metrics (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:39439] } background_task = dns_servers_internal dns_group = internal
20282024-12-13T21:18:23.966ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 54.42428ms iteration = 2 status = {"addresses":["[::1]:39439"]}
20292024-12-13T21:18:23.966ZINFOtest_mgs_metrics (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:41531] } background_task = dns_servers_external dns_group = external
20302024-12-13T21:18:23.966ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 53.424029ms iteration = 2 status = {"addresses":["[::1]:41531"]}
20312024-12-13T21:18:23.966ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
20322024-12-13T21:18:23.966ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config
20332024-12-13T21:18:23.967ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 32.98\u{b5}s iteration = 2 status = {"error":"no config"}
20342024-12-13T21:18:23.967ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
20352024-12-13T21:18:23.967ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config
20362024-12-13T21:18:23.967ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 20.241\u{b5}s iteration = 2 status = {"error":"no config"}
20372024-12-13T21:18:23.967ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
20382024-12-13T21:18:23.967ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20392024-12-13T21:18:23.967ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20402024-12-13T21:18:23.967ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20412024-12-13T21:18:23.967ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20422024-12-13T21:18:23.979ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20432024-12-13T21:18:23.986ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20442024-12-13T21:18:23.991ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20452024-12-13T21:18:23.991ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20462024-12-13T21:18:23.991ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20472024-12-13T21:18:23.992ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20482024-12-13T21:18:24.005ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
20492024-12-13T21:18:24.005ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20502024-12-13T21:18:24.006ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20512024-12-13T21:18:24.006ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20522024-12-13T21:18:24.038ZDEBGtest_mgs_metrics (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
20532024-12-13T21:18:24.038ZDEBGtest_mgs_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5d2c9bf5-7041-43f1-a48f-12ee21b068fc dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
20542024-12-13T21:18:24.038ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
20552024-12-13T21:18:24.038ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20562024-12-13T21:18:24.038ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20572024-12-13T21:18:24.039ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20582024-12-13T21:18:24.040ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20592024-12-13T21:18:24.041ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20602024-12-13T21:18:24.049ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
20612024-12-13T21:18:24.049ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
20622024-12-13T21:18:24.049ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
20632024-12-13T21:18:24.050ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
20642024-12-13T21:18:24.059ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20652024-12-13T21:18:24.070ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20662024-12-13T21:18:24.070ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20672024-12-13T21:18:24.070ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20682024-12-13T21:18:24.071ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20692024-12-13T21:18:24.071ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
20702024-12-13T21:18:24.094ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
20712024-12-13T21:18:24.095ZDEBGtest_mgs_metrics (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
20722024-12-13T21:18:24.096ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
20732024-12-13T21:18:24.096ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
20742024-12-13T21:18:24.096ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20752024-12-13T21:18:24.097ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20762024-12-13T21:18:24.097ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20772024-12-13T21:18:24.097ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20782024-12-13T21:18:24.097ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20792024-12-13T21:18:24.103ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: fa 5b 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .[..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56151 req_id = 3538d118-036b-4c2e-b053-06eb160b9b67
20802024-12-13T21:18:24.110ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64091,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56151 req_id = 3538d118-036b-4c2e-b053-06eb160b9b67
20812024-12-13T21:18:24.111ZDEBGtest_mgs_metrics (store): query key key = _mgs._tcp
20822024-12-13T21:18:24.112ZDEBGtest_mgs_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
20832024-12-13T21:18:24.112ZDEBGtest_mgs_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
20842024-12-13T21:18:24.112ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56151 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41771, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39497, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3538d118-036b-4c2e-b053-06eb160b9b67
20852024-12-13T21:18:24.112ZDEBGtest_mgs_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5d2c9bf5-7041-43f1-a48f-12ee21b068fc dns_zone_name = control-plane.oxide.internal found_so_far = 18 version = 1
20862024-12-13T21:18:24.112ZDEBGtest_mgs_metrics (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
20872024-12-13T21:18:24.112ZINFOtest_mgs_metrics (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal generation = 1
20882024-12-13T21:18:24.113ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 207.72407ms iteration = 2 status = {"generation":1}
20892024-12-13T21:18:24.113ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
20902024-12-13T21:18:24.194ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:39439 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:39439] } uri = http://[::1]:39439/config
20912024-12-13T21:18:24.195ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
20922024-12-13T21:18:24.195ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20932024-12-13T21:18:24.195ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20942024-12-13T21:18:24.195ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20952024-12-13T21:18:24.196ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20962024-12-13T21:18:24.205ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20972024-12-13T21:18:24.211ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20982024-12-13T21:18:24.212ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20992024-12-13T21:18:24.212ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21002024-12-13T21:18:24.217ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21012024-12-13T21:18:24.217ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21022024-12-13T21:18:24.223ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
21032024-12-13T21:18:24.226ZINFOtest_mgs_metrics (nexus): Rack initialized
21042024-12-13T21:18:24.227ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21052024-12-13T21:18:24.227ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21062024-12-13T21:18:24.228ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21072024-12-13T21:18:24.229ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 41771, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39497, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2099, tv_nsec: 11461397 } })
21082024-12-13T21:18:24.230ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
21092024-12-13T21:18:24.230ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
21102024-12-13T21:18:24.230ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:39439 remote_addr = [::1]:55270
21112024-12-13T21:18:24.232ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21122024-12-13T21:18:24.242ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21132024-12-13T21:18:24.242ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
21142024-12-13T21:18:24.242ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21152024-12-13T21:18:24.243ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21162024-12-13T21:18:24.246ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:39439 method = PUT remote_addr = [::1]:55270 req_id = 5368c97d-07b0-4f7e-b4f2-a5985d89b25d uri = /config
21172024-12-13T21:18:24.258ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 1 req_id = 5368c97d-07b0-4f7e-b4f2-a5985d89b25d
21182024-12-13T21:18:24.260ZINFOtest_mgs_metrics (store): updated generation new_generation = 1 req_id = 5368c97d-07b0-4f7e-b4f2-a5985d89b25d
21192024-12-13T21:18:24.260ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 14218 local_addr = [::1]:39439 method = PUT remote_addr = [::1]:55270 req_id = 5368c97d-07b0-4f7e-b4f2-a5985d89b25d response_code = 204 uri = /config
21202024-12-13T21:18:24.261ZDEBGtest_mgs_metrics (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
21212024-12-13T21:18:24.261ZDEBGtest_mgs_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 19102c31-841e-4129-aa93-975ade989571 dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
21222024-12-13T21:18:24.261ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
21232024-12-13T21:18:24.261ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21242024-12-13T21:18:24.261ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
21252024-12-13T21:18:24.262ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21262024-12-13T21:18:24.262ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21272024-12-13T21:18:24.276ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 41 75 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e Au..........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59751 req_id = 812ef400-6232-4ba1-804c-776ae4a6276e
21282024-12-13T21:18:24.276ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16757,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59751 req_id = 812ef400-6232-4ba1-804c-776ae4a6276e
21292024-12-13T21:18:24.276ZDEBGtest_mgs_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
21302024-12-13T21:18:24.277ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59751 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 812ef400-6232-4ba1-804c-776ae4a6276e
21312024-12-13T21:18:24.277ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5f 40 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e _@..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61541 req_id = 53f2118b-17bf-4f40-8386-a2ed5dfb6a56
21322024-12-13T21:18:24.278ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24384,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61541 req_id = 53f2118b-17bf-4f40-8386-a2ed5dfb6a56
21332024-12-13T21:18:24.278ZDEBGtest_mgs_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
21342024-12-13T21:18:24.278ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61541 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 53f2118b-17bf-4f40-8386-a2ed5dfb6a56
21352024-12-13T21:18:24.278ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:39439 generation = 1 result = Ok(Response { url: "http://[::1]:39439/config", status: 204, headers: {"x-request-id": "5368c97d-07b0-4f7e-b4f2-a5985d89b25d", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} }) servers = DnsServersList { addresses: [[::1]:39439] }
21362024-12-13T21:18:24.278ZINFOtest_mgs_metrics (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:39439 generation = 1 servers = DnsServersList { addresses: [[::1]:39439] }
21372024-12-13T21:18:24.279ZINFOtest_mgs_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:39439] }
21382024-12-13T21:18:24.282ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 166.235403ms iteration = 3 status = {"generation":1,"server_results":{"[::1]:39439":{"Ok":null}}}
21392024-12-13T21:18:24.282ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21402024-12-13T21:18:24.282ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21412024-12-13T21:18:24.282ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21422024-12-13T21:18:24.382ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
21432024-12-13T21:18:24.386ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21442024-12-13T21:18:24.418ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21452024-12-13T21:18:24.418ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
21462024-12-13T21:18:24.418ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21472024-12-13T21:18:24.418ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21482024-12-13T21:18:24.430ZINFOtest_mgs_metrics (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-13T21:18:16.610333Z, time_modified: 2024-12-13T21:18:16.610333Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 00f97f81-909e-4b96-81e4-53ffb6a12150, db_silo: Silo { identity: SiloIdentity { id: 00f97f81-909e-4b96-81e4-53ffb6a12150, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-12-13T21:18:23.022425Z, time_modified: 2024-12-13T21:18:23.022425Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: 00f97f81-909e-4b96-81e4-53ffb6a12150, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 00f97f81-909e-4b96-81e4-53ffb6a12150, db_silo: Silo { identity: SiloIdentity { id: 00f97f81-909e-4b96-81e4-53ffb6a12150, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-12-13T21:18:23.022425Z, time_modified: 2024-12-13T21:18:23.022425Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
21492024-12-13T21:18:24.430ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 522.121625ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"00f97f81-909e-4b96-81e4-53ffb6a12150","tls_certs":[]}},"default_endpoint":{"silo_id":"00f97f81-909e-4b96-81e4-53ffb6a12150","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo 00f97f81-909e-4b96-81e4-53ffb6a12150 with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
21502024-12-13T21:18:24.430ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 89 be 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60113 req_id = 91b82be1-5aff-4fac-bb9b-8a9be0070cc8
21512024-12-13T21:18:24.431ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35262,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60113 req_id = 91b82be1-5aff-4fac-bb9b-8a9be0070cc8
21522024-12-13T21:18:24.432ZDEBGtest_mgs_metrics (store): query key key = _clickhouse-admin-keeper._tcp
21532024-12-13T21:18:24.432ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 35262, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60113 req_id = 91b82be1-5aff-4fac-bb9b-8a9be0070cc8
21542024-12-13T21:18:24.436ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21552024-12-13T21:18:24.437ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21562024-12-13T21:18:24.437ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21572024-12-13T21:18:24.437ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection background_task = inventory_collection
21582024-12-13T21:18:24.438ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:41771
21592024-12-13T21:18:24.438ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/ignition
21602024-12-13T21:18:24.444ZDEBGtest_mgs_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 19102c31-841e-4129-aa93-975ade989571 dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
21612024-12-13T21:18:24.444ZDEBGtest_mgs_metrics (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
21622024-12-13T21:18:24.445ZINFOtest_mgs_metrics (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external generation = 2
21632024-12-13T21:18:24.445ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 538.375308ms iteration = 2 status = {"generation":2}
21642024-12-13T21:18:24.445ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
21652024-12-13T21:18:24.489ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:41531 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:41531] } uri = http://[::1]:41531/config
21662024-12-13T21:18:24.494ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:41531 remote_addr = [::1]:34078
21672024-12-13T21:18:24.500ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21682024-12-13T21:18:24.500ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21692024-12-13T21:18:24.500ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21702024-12-13T21:18:24.500ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21712024-12-13T21:18:24.500ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21722024-12-13T21:18:24.501ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:41531 method = PUT remote_addr = [::1]:34078 req_id = 27251ab2-ab8f-4873-9b46-8c0bd7dad234 uri = /config
21732024-12-13T21:18:24.503ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 2 req_id = 27251ab2-ab8f-4873-9b46-8c0bd7dad234
21742024-12-13T21:18:24.503ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
21752024-12-13T21:18:24.503ZDEBGtest_mgs_metrics (store): creating tree tree_name = generation_2_zone_oxide-dev.test
21762024-12-13T21:18:24.507ZDEBGtest_mgs_metrics (store): updating current config
21772024-12-13T21:18:24.507ZDEBGtest_mgs_metrics (store): flushing default tree
21782024-12-13T21:18:24.507ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21792024-12-13T21:18:24.508ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21802024-12-13T21:18:24.508ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21812024-12-13T21:18:24.509ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21822024-12-13T21:18:24.509ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8da61e69-f188-4358-9ede-eb9184c462fe", "content-length": "835", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
21832024-12-13T21:18:24.510ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0
21842024-12-13T21:18:24.511ZINFOtest_mgs_metrics (store): pruning trees for generations older than 2 keep = 3
21852024-12-13T21:18:24.511ZINFOtest_mgs_metrics (store): updated generation new_generation = 2 req_id = 27251ab2-ab8f-4873-9b46-8c0bd7dad234
21862024-12-13T21:18:24.511ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 10348 local_addr = [::1]:41531 method = PUT remote_addr = [::1]:34078 req_id = 27251ab2-ab8f-4873-9b46-8c0bd7dad234 response_code = 204 uri = /config
21872024-12-13T21:18:24.523ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:41531 generation = 2 result = Ok(Response { url: "http://[::1]:41531/config", status: 204, headers: {"x-request-id": "27251ab2-ab8f-4873-9b46-8c0bd7dad234", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} }) servers = DnsServersList { addresses: [[::1]:41531] }
21882024-12-13T21:18:24.524ZINFOtest_mgs_metrics (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:41531 generation = 2 servers = DnsServersList { addresses: [[::1]:41531] }
21892024-12-13T21:18:24.526ZINFOtest_mgs_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:41531] }
21902024-12-13T21:18:24.527ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 81.092084ms iteration = 3 status = {"generation":2,"server_results":{"[::1]:41531":{"Ok":null}}}
21912024-12-13T21:18:24.527ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21922024-12-13T21:18:24.528ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21932024-12-13T21:18:24.528ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21942024-12-13T21:18:24.528ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21952024-12-13T21:18:24.545ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21962024-12-13T21:18:24.555ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21972024-12-13T21:18:24.555ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21982024-12-13T21:18:24.555ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21992024-12-13T21:18:24.556ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22002024-12-13T21:18:24.557ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22012024-12-13T21:18:24.557ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22022024-12-13T21:18:24.557ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22032024-12-13T21:18:24.557ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f7ebc54-f472-4d79-9e2c-23b2d70949cf", "content-length": "734", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22042024-12-13T21:18:24.558ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/sp/caboose?firmware_slot=0
22052024-12-13T21:18:24.563ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22062024-12-13T21:18:24.563ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22072024-12-13T21:18:24.564ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22082024-12-13T21:18:24.565ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22092024-12-13T21:18:24.566ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6b4eb59-d3a2-46b5-aa32-5dded4d90165", "content-length": "111", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22102024-12-13T21:18:24.567ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/sp/caboose?firmware_slot=1
22112024-12-13T21:18:24.568ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22122024-12-13T21:18:24.568ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22132024-12-13T21:18:24.568ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22142024-12-13T21:18:24.568ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22152024-12-13T21:18:24.571ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22162024-12-13T21:18:24.583ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
22172024-12-13T21:18:24.583ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22182024-12-13T21:18:24.583ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22192024-12-13T21:18:24.584ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22202024-12-13T21:18:24.588ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2c59c9f-3d17-41fa-92ef-d72ffaa0a183", "content-length": "111", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22212024-12-13T21:18:24.588ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/rot/caboose?firmware_slot=0
22222024-12-13T21:18:24.590ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22232024-12-13T21:18:24.593ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22242024-12-13T21:18:24.593ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22252024-12-13T21:18:24.595ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ae514e2-abc8-4e1e-b3a0-33b81c95c5fa", "content-length": "167", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22262024-12-13T21:18:24.595ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/rot/caboose?firmware_slot=1
22272024-12-13T21:18:24.597ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22282024-12-13T21:18:24.597ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22292024-12-13T21:18:24.599ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22302024-12-13T21:18:24.603ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf7d546d-6c04-455d-a6aa-017702ed6057", "content-length": "167", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22312024-12-13T21:18:24.603ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/stage0/caboose?firmware_slot=0
22322024-12-13T21:18:24.620ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22332024-12-13T21:18:24.620ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22342024-12-13T21:18:24.621ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22352024-12-13T21:18:24.627ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
22362024-12-13T21:18:24.627ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22372024-12-13T21:18:24.627ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22382024-12-13T21:18:24.627ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22392024-12-13T21:18:24.629ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22402024-12-13T21:18:24.630ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "57f3ef51-1d91-4403-ac63-42dbcff06516", "content-length": "175", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22412024-12-13T21:18:24.633ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/stage0/caboose?firmware_slot=1
22422024-12-13T21:18:24.638ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22432024-12-13T21:18:24.638ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22442024-12-13T21:18:24.638ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22452024-12-13T21:18:24.639ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22462024-12-13T21:18:24.641ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c4e1353-b95b-4a8a-98ce-412a137f45ad", "content-length": "175", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22472024-12-13T21:18:24.641ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/rot/cmpa
22482024-12-13T21:18:24.650ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22492024-12-13T21:18:24.651ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22502024-12-13T21:18:24.652ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22512024-12-13T21:18:24.659ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22522024-12-13T21:18:24.660ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d77a4425-10e2-4816-b767-9b061fb0737f", "content-length": "702", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22532024-12-13T21:18:24.661ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/rot/cfpa
22542024-12-13T21:18:24.661ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22552024-12-13T21:18:24.670ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
22562024-12-13T21:18:24.671ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
22572024-12-13T21:18:24.671ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22582024-12-13T21:18:24.671ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22592024-12-13T21:18:24.672ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22602024-12-13T21:18:24.672ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2f0a9a0-7557-4dd6-a1df-7f27a146b56b", "content-length": "727", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22612024-12-13T21:18:24.673ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/rot/cfpa
22622024-12-13T21:18:24.674ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22632024-12-13T21:18:24.674ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22642024-12-13T21:18:24.674ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22652024-12-13T21:18:24.675ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a266945-8737-4a16-898f-c94eb0e0ed87", "content-length": "729", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22662024-12-13T21:18:24.676ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/0/component/rot/cfpa
22672024-12-13T21:18:24.676ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "761ae185-27ba-4f6e-82c3-0e6ed57e7015", "content-length": "728", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22682024-12-13T21:18:24.676ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1
22692024-12-13T21:18:24.677ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22702024-12-13T21:18:24.678ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22712024-12-13T21:18:24.678ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22722024-12-13T21:18:24.679ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "705cdd48-c9e6-41dc-9cc9-9f1b20d84bf9", "content-length": "734", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22732024-12-13T21:18:24.679ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/sp/caboose?firmware_slot=0
22742024-12-13T21:18:24.681ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22752024-12-13T21:18:24.681ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22762024-12-13T21:18:24.682ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22772024-12-13T21:18:24.682ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f35ac6a9-a617-4008-b4f9-c59ecb73dca0", "content-length": "111", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22782024-12-13T21:18:24.682ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/sp/caboose?firmware_slot=1
22792024-12-13T21:18:24.684ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5da81895-0187-4acf-ae58-ef8a10b599e6", "content-length": "111", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22802024-12-13T21:18:24.685ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/rot/caboose?firmware_slot=0
22812024-12-13T21:18:24.685ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22822024-12-13T21:18:24.685ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22832024-12-13T21:18:24.685ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22842024-12-13T21:18:24.685ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22852024-12-13T21:18:24.686ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22862024-12-13T21:18:24.688ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62939860-76a0-48e3-80fb-6cc7c23cee23", "content-length": "167", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22872024-12-13T21:18:24.688ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/rot/caboose?firmware_slot=1
22882024-12-13T21:18:24.688ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22892024-12-13T21:18:24.689ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22902024-12-13T21:18:24.689ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22912024-12-13T21:18:24.689ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22922024-12-13T21:18:24.691ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3b66ac1-897b-41c1-95e2-f10f4057cff2", "content-length": "167", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
22932024-12-13T21:18:24.692ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/stage0/caboose?firmware_slot=0
22942024-12-13T21:18:24.692ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22952024-12-13T21:18:24.692ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22962024-12-13T21:18:24.692ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22972024-12-13T21:18:24.693ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22982024-12-13T21:18:24.695ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22992024-12-13T21:18:24.704ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23002024-12-13T21:18:24.705ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
23012024-12-13T21:18:24.705ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
23022024-12-13T21:18:24.705ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23032024-12-13T21:18:24.705ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23042024-12-13T21:18:24.705ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23052024-12-13T21:18:24.705ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23062024-12-13T21:18:24.705ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d161ab32-9c60-4aa1-b807-ac72911a33d6", "content-length": "175", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23072024-12-13T21:18:24.705ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/stage0/caboose?firmware_slot=1
23082024-12-13T21:18:24.707ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23092024-12-13T21:18:24.708ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23102024-12-13T21:18:24.708ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23112024-12-13T21:18:24.709ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23122024-12-13T21:18:24.709ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f1d2aac-2baa-4b88-887f-712717fa4125", "content-length": "175", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23132024-12-13T21:18:24.710ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/rot/cmpa
23142024-12-13T21:18:24.711ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c799fa5-93c1-4521-8f9e-86866e5dc19e", "content-length": "702", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23152024-12-13T21:18:24.711ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/rot/cfpa
23162024-12-13T21:18:24.712ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23172024-12-13T21:18:24.712ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23182024-12-13T21:18:24.712ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23192024-12-13T21:18:24.713ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23202024-12-13T21:18:24.714ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe2a4960-4d4e-4b6f-b405-5cb139a1849e", "content-length": "727", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23212024-12-13T21:18:24.715ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/rot/cfpa
23222024-12-13T21:18:24.716ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c18096f-9e97-4843-99a3-ae3a05b206ae", "content-length": "729", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23232024-12-13T21:18:24.716ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/switch/1/component/rot/cfpa
23242024-12-13T21:18:24.717ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23252024-12-13T21:18:24.726ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23262024-12-13T21:18:24.726ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23272024-12-13T21:18:24.726ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23282024-12-13T21:18:24.726ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23292024-12-13T21:18:24.729ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a54acea-8b84-455d-b49d-769086200d8f", "content-length": "728", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23302024-12-13T21:18:24.729ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0
23312024-12-13T21:18:24.730ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfd03049-3b0e-4e84-917e-a3e3e2ef2c91", "content-length": "723", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23322024-12-13T21:18:24.732ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/sp/caboose?firmware_slot=0
23332024-12-13T21:18:24.732ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23342024-12-13T21:18:24.732ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23352024-12-13T21:18:24.733ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23362024-12-13T21:18:24.743ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23372024-12-13T21:18:24.743ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23382024-12-13T21:18:24.743ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23392024-12-13T21:18:24.745ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9644578f-ce70-4806-a6d4-d9af3507b4d6", "content-length": "109", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23402024-12-13T21:18:24.746ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/sp/caboose?firmware_slot=1
23412024-12-13T21:18:24.746ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23422024-12-13T21:18:24.746ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23432024-12-13T21:18:24.746ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23442024-12-13T21:18:24.748ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc761cb3-4728-4025-8b9c-c98bac6eb907", "content-length": "109", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23452024-12-13T21:18:24.749ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/rot/caboose?firmware_slot=0
23462024-12-13T21:18:24.749ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23472024-12-13T21:18:24.749ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23482024-12-13T21:18:24.750ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23492024-12-13T21:18:24.752ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
23502024-12-13T21:18:24.752ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23512024-12-13T21:18:24.753ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23522024-12-13T21:18:24.753ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23532024-12-13T21:18:24.754ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23542024-12-13T21:18:24.756ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23552024-12-13T21:18:24.756ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23562024-12-13T21:18:24.756ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23572024-12-13T21:18:24.757ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23582024-12-13T21:18:24.759ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f33f30b-8f74-46db-9419-8c6f85cab342", "content-length": "169", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23592024-12-13T21:18:24.759ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/rot/caboose?firmware_slot=1
23602024-12-13T21:18:24.759ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23612024-12-13T21:18:24.759ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23622024-12-13T21:18:24.759ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23632024-12-13T21:18:24.760ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23642024-12-13T21:18:24.762ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23652024-12-13T21:18:24.771ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23662024-12-13T21:18:24.771ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
23672024-12-13T21:18:24.771ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
23682024-12-13T21:18:24.771ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23692024-12-13T21:18:24.771ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23702024-12-13T21:18:24.771ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23712024-12-13T21:18:24.771ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23722024-12-13T21:18:24.773ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98bb6d2e-13b4-4f15-9eb2-ae856913822d", "content-length": "169", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23732024-12-13T21:18:24.774ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/stage0/caboose?firmware_slot=0
23742024-12-13T21:18:24.775ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85233d3f-5ad6-4e1c-b1ae-830baff0d329", "content-length": "178", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23752024-12-13T21:18:24.775ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/stage0/caboose?firmware_slot=1
23762024-12-13T21:18:24.777ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23772024-12-13T21:18:24.777ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23782024-12-13T21:18:24.777ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23792024-12-13T21:18:24.777ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23802024-12-13T21:18:24.779ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ddddf87-7f90-4e94-8f31-22e7a42609e7", "content-length": "178", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23812024-12-13T21:18:24.779ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/rot/cmpa
23822024-12-13T21:18:24.780ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f2ec1b3-d4b9-4557-84c4-79910b0219f7", "content-length": "702", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23832024-12-13T21:18:24.780ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/rot/cfpa
23842024-12-13T21:18:24.780ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23852024-12-13T21:18:24.780ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23862024-12-13T21:18:24.780ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23872024-12-13T21:18:24.781ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23882024-12-13T21:18:24.783ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "79b5d3fe-c013-47b1-9e1a-da1d08b33e59", "content-length": "727", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23892024-12-13T21:18:24.784ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/rot/cfpa
23902024-12-13T21:18:24.784ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23912024-12-13T21:18:24.792ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23922024-12-13T21:18:24.792ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23932024-12-13T21:18:24.792ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23942024-12-13T21:18:24.794ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23952024-12-13T21:18:24.794ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7626bb6a-0d68-4128-ae26-6938b291bdaf", "content-length": "729", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23962024-12-13T21:18:24.795ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/0/component/rot/cfpa
23972024-12-13T21:18:24.796ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0f9f9f4-1ea1-4eb2-9856-c99fac0aef06", "content-length": "728", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
23982024-12-13T21:18:24.796ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1
23992024-12-13T21:18:24.798ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e77864f0-fe23-44f7-bb39-56d12269cc3f", "content-length": "723", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24002024-12-13T21:18:24.798ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/sp/caboose?firmware_slot=0
24012024-12-13T21:18:24.800ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b28c5c9b-8a5a-4ec2-8d81-50b1143d9c90", "content-length": "109", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24022024-12-13T21:18:24.800ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/sp/caboose?firmware_slot=1
24032024-12-13T21:18:24.801ZDEBGtest_mgs_metrics (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 07b099d9-af6c-4a52-8be7-26d4b3e67527, kind: Service { id: ab3c6976-382d-4564-bfcf-cb6a75ba0c83 }, name: Name("external-dns-ab3c6976-382d-4564-bfcf-cb6a75ba0c83"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 184b8d5c-4a61-45dd-a4eb-4a4a6b73719b, kind: Service { id: 97039868-0386-49ec-9259-75c36edb1ac3 }, name: Name("nexus-97039868-0386-49ec-9259-75c36edb1ac3"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
24042024-12-13T21:18:24.801ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
24052024-12-13T21:18:24.801ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24062024-12-13T21:18:24.802ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24072024-12-13T21:18:24.802ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24082024-12-13T21:18:24.802ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24092024-12-13T21:18:24.804ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7ba4389-310a-452a-a6c6-68610d331b28", "content-length": "109", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24102024-12-13T21:18:24.804ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/rot/caboose?firmware_slot=0
24112024-12-13T21:18:24.806ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9285392a-591b-4a5b-af5b-3cbf08cd8a89", "content-length": "169", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24122024-12-13T21:18:24.806ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/rot/caboose?firmware_slot=1
24132024-12-13T21:18:24.806ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24142024-12-13T21:18:24.811ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
24152024-12-13T21:18:24.811ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24162024-12-13T21:18:24.811ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24172024-12-13T21:18:24.812ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24182024-12-13T21:18:24.814ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8950f3b5-a470-4694-88aa-c2221437d991", "content-length": "169", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24192024-12-13T21:18:24.815ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/stage0/caboose?firmware_slot=0
24202024-12-13T21:18:24.815ZTRACtest_mgs_metrics (InternalApi): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-12-13T20:56:30.038685Z, time_modified: 2024-12-13T21:18:23.291139Z, allowed_ips: None }
24212024-12-13T21:18:24.815ZDEBGtest_mgs_metrics (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
24222024-12-13T21:18:24.815ZDEBGtest_mgs_metrics (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 07b099d9-af6c-4a52-8be7-26d4b3e67527, kind: Service { id: ab3c6976-382d-4564-bfcf-cb6a75ba0c83 }, name: Name("external-dns-ab3c6976-382d-4564-bfcf-cb6a75ba0c83"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 184b8d5c-4a61-45dd-a4eb-4a4a6b73719b, kind: Service { id: 97039868-0386-49ec-9259-75c36edb1ac3 }, name: Name("nexus-97039868-0386-49ec-9259-75c36edb1ac3"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
24232024-12-13T21:18:24.817ZDEBGtest_mgs_metrics (InternalApi): resolved 2 rules for sleds
24242024-12-13T21:18:24.817ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4419ba3d-2979-40c6-a29d-9531ddd58470", "content-length": "178", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24252024-12-13T21:18:24.818ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/stage0/caboose?firmware_slot=1
24262024-12-13T21:18:24.820ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "97877fee-5d45-4bd8-aa2c-34cc0431a6dc", "content-length": "178", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24272024-12-13T21:18:24.820ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/rot/cmpa
24282024-12-13T21:18:24.820ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9efa670a-965d-4007-9679-a0537c045d49", "content-length": "702", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24292024-12-13T21:18:24.821ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/rot/cfpa
24302024-12-13T21:18:24.822ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "24faee01-5674-4dae-abf7-5ff5b6a46839", "content-length": "727", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24312024-12-13T21:18:24.822ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/rot/cfpa
24322024-12-13T21:18:24.823ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c87fb9f1-0f7b-4e6d-b8a8-d15b41fcf6a6", "content-length": "729", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24332024-12-13T21:18:24.823ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:41771 method = GET uri = http://[::1]:41771/sp/sled/1/component/rot/cfpa
24342024-12-13T21:18:24.824ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:41771 result = Ok(Response { url: "http://[::1]:41771/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0e7dd5d-eeab-4f49-96c6-1b31fb3975c8", "content-length": "728", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24352024-12-13T21:18:24.824ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:39497
24362024-12-13T21:18:24.824ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:39497 method = GET uri = http://[::1]:39497/ignition
24372024-12-13T21:18:24.825ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:39497 result = Ok(Response { url: "http://[::1]:39497/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "956691a6-36c0-434b-98a9-3d174c748e22", "content-length": "835", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24382024-12-13T21:18:24.825ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:39497 method = GET uri = http://[::1]:39497/sp/switch/0
24392024-12-13T21:18:24.827ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:39497 result = Ok(Response { url: "http://[::1]:39497/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "486b02bd-13ef-4424-8f9a-b04d0673aa0f", "content-length": "734", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24402024-12-13T21:18:24.827ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:39497 method = GET uri = http://[::1]:39497/sp/switch/1
24412024-12-13T21:18:24.828ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:39497 result = Ok(Response { url: "http://[::1]:39497/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f39a05b-042e-4520-aa2f-b52b145542b9", "content-length": "734", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24422024-12-13T21:18:24.828ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:39497 method = GET uri = http://[::1]:39497/sp/sled/0
24432024-12-13T21:18:24.829ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:39497 result = Ok(Response { url: "http://[::1]:39497/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "811e2651-9f9b-4d01-917b-7afc6029afb5", "content-length": "723", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24442024-12-13T21:18:24.829ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:39497 method = GET uri = http://[::1]:39497/sp/sled/1
24452024-12-13T21:18:24.831ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:39497 result = Ok(Response { url: "http://[::1]:39497/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "edeeb0a4-4c4b-4db3-a71b-6690be625bcb", "content-length": "723", "date": "Fri, 13 Dec 2024 21:18:23 GMT"} })
24462024-12-13T21:18:24.831ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
24472024-12-13T21:18:24.831ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24482024-12-13T21:18:24.831ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24492024-12-13T21:18:24.831ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24502024-12-13T21:18:24.832ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24512024-12-13T21:18:24.835ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24522024-12-13T21:18:24.841ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24532024-12-13T21:18:24.841ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
24542024-12-13T21:18:24.842ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24552024-12-13T21:18:24.842ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24562024-12-13T21:18:24.842ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24572024-12-13T21:18:24.842ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24582024-12-13T21:18:24.849ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24592024-12-13T21:18:24.853ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24602024-12-13T21:18:24.853ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24612024-12-13T21:18:24.853ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24622024-12-13T21:18:24.854ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24632024-12-13T21:18:24.909ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:42715
24642024-12-13T21:18:24.909ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = http://[::1]:42715 background_task = inventory_collection body = None method = GET uri = http://[::1]:42715/inventory
24652024-12-13T21:18:24.910ZDEBGtest_mgs_metrics (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2024-12-13T21:18:18.665137Z, time_modified: 2024-12-13T21:18:18.665137Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(45365), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
24662024-12-13T21:18:24.910ZDEBGtest_mgs_metrics (InternalApi): sending firewall rules to sled agents
24672024-12-13T21:18:24.910ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24682024-12-13T21:18:24.910ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24692024-12-13T21:18:24.911ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24702024-12-13T21:18:24.911ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:42715 remote_addr = [::1]:41964 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
24712024-12-13T21:18:24.912ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:42715 method = GET remote_addr = [::1]:41964 req_id = cb2c8cbc-4ae4-4c94-a1e4-3b7dc4d511db sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
24722024-12-13T21:18:24.912ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 520 local_addr = [::1]:42715 method = GET remote_addr = [::1]:41964 req_id = cb2c8cbc-4ae4-4c94-a1e4-3b7dc4d511db response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
24732024-12-13T21:18:24.913ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = http://[::1]:42715 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:42715/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb2c8cbc-4ae4-4c94-a1e4-3b7dc4d511db", "content-length": "4237", "date": "Fri, 13 Dec 2024 21:18:24 GMT"} })
24742024-12-13T21:18:24.974ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:45365
24752024-12-13T21:18:24.975ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = http://[::1]:45365 background_task = inventory_collection body = None method = GET uri = http://[::1]:45365/inventory
24762024-12-13T21:18:24.976ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
24772024-12-13T21:18:24.976ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24782024-12-13T21:18:24.976ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24792024-12-13T21:18:24.976ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24802024-12-13T21:18:24.990ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24812024-12-13T21:18:24.990ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45365 remote_addr = [::1]:41858 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
24822024-12-13T21:18:24.991ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45365 method = GET remote_addr = [::1]:41858 req_id = 114d1b55-f300-4154-8d85-abb6c23914ff sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
24832024-12-13T21:18:24.992ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 790 local_addr = [::1]:45365 method = GET remote_addr = [::1]:41858 req_id = 114d1b55-f300-4154-8d85-abb6c23914ff response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
24842024-12-13T21:18:24.993ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = http://[::1]:45365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:45365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "114d1b55-f300-4154-8d85-abb6c23914ff", "content-length": "6315", "date": "Fri, 13 Dec 2024 21:18:24 GMT"} })
24852024-12-13T21:18:24.994ZDEBGtest_mgs_metrics (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
24862024-12-13T21:18:24.994ZDEBGtest_mgs_metrics (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
24872024-12-13T21:18:24.994ZDEBGtest_mgs_metrics (BackgroundTasks): finished collection background_task = inventory_collection
24882024-12-13T21:18:24.996ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
24892024-12-13T21:18:24.997ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24902024-12-13T21:18:24.997ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24912024-12-13T21:18:24.997ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24922024-12-13T21:18:24.998ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24932024-12-13T21:18:24.998ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24942024-12-13T21:18:25.004ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
24952024-12-13T21:18:25.086ZDEBGtest_mgs_metrics (InternalApi): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:45365/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
24962024-12-13T21:18:25.091ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45365 remote_addr = [::1]:41860 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
24972024-12-13T21:18:25.092ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24982024-12-13T21:18:25.096ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
24992024-12-13T21:18:25.097ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
25002024-12-13T21:18:25.097ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
25012024-12-13T21:18:25.098ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
25022024-12-13T21:18:25.101ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41860 req_id = 48d68049-f09a-4dad-bcf9-bc8b5acc32b6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
25032024-12-13T21:18:25.103ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 1304 local_addr = [::1]:45365 method = PUT remote_addr = [::1]:41860 req_id = 48d68049-f09a-4dad-bcf9-bc8b5acc32b6 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
25042024-12-13T21:18:25.108ZDEBGtest_mgs_metrics (InternalApi): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:45365/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "48d68049-f09a-4dad-bcf9-bc8b5acc32b6", "date": "Fri, 13 Dec 2024 21:18:24 GMT"} })
25052024-12-13T21:18:25.110ZINFOtest_mgs_metrics (InternalApi): updated firewall rules on 1 sleds
25062024-12-13T21:18:25.110ZINFOtest_mgs_metrics (nexus): plumbed initial IP allowlist
25072024-12-13T21:18:25.145ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = / versions = all
25082024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /access versions = all
25092024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /assets/{path} versions = all
25102024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /device/auth versions = all
25112024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /device/confirm versions = all
25122024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /device/success versions = all
25132024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /device/token versions = all
25142024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /device/verify versions = all
25152024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /experimental/v1/probes versions = all
25162024-12-13T21:18:25.146ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /experimental/v1/probes versions = all
25172024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /experimental/v1/probes/{probe} versions = all
25182024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /experimental/v1/probes/{probe} versions = all
25192024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /images versions = all
25202024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /login versions = all
25212024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /login/{silo_name}/local versions = all
25222024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
25232024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
25242024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
25252024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /lookup/{path} versions = all
25262024-12-13T21:18:25.147ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /projects/{path} versions = all
25272024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /projects-new versions = all
25282024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /settings/{path} versions = all
25292024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /system/{path} versions = all
25302024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /utilization versions = all
25312024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/certificates versions = all
25322024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/certificates versions = all
25332024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/certificates/{certificate} versions = all
25342024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/certificates/{certificate} versions = all
25352024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/disks versions = all
25362024-12-13T21:18:25.148ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/disks versions = all
25372024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/disks/{disk} versions = all
25382024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/disks/{disk} versions = all
25392024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/disks/{disk}/bulk-write versions = all
25402024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
25412024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
25422024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/disks/{disk}/finalize versions = all
25432024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
25442024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/floating-ips versions = all
25452024-12-13T21:18:25.149ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/floating-ips versions = all
25462024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
25472024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/floating-ips/{floating_ip} versions = all
25482024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
25492024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
25502024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
25512024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/groups versions = all
25522024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/groups/{group_id} versions = all
25532024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/images versions = all
25542024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/images versions = all
25552024-12-13T21:18:25.150ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/images/{image} versions = all
25562024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/images/{image} versions = all
25572024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/images/{image}/demote versions = all
25582024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/images/{image}/promote versions = all
25592024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/instances versions = all
25602024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/instances versions = all
25612024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/instances/{instance} versions = all
25622024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/instances/{instance} versions = all
25632024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/instances/{instance} versions = all
25642024-12-13T21:18:25.151ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/instances/{instance}/disks versions = all
25652024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/instances/{instance}/disks/attach versions = all
25662024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/instances/{instance}/disks/detach versions = all
25672024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/instances/{instance}/external-ips versions = all
25682024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
25692024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
25702024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/instances/{instance}/reboot versions = all
25712024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/instances/{instance}/serial-console versions = all
25722024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
25732024-12-13T21:18:25.152ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
25742024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/instances/{instance}/start versions = all
25752024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/instances/{instance}/stop versions = all
25762024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/internet-gateway-ip-addresses versions = all
25772024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/internet-gateway-ip-addresses versions = all
25782024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
25792024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/internet-gateway-ip-pools versions = all
25802024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/internet-gateway-ip-pools versions = all
25812024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
25822024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/internet-gateways versions = all
25832024-12-13T21:18:25.153ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/internet-gateways versions = all
25842024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
25852024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/internet-gateways/{gateway} versions = all
25862024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/ip-pools versions = all
25872024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/ip-pools/{pool} versions = all
25882024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/login/{silo_name}/local versions = all
25892024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/logout versions = all
25902024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/me versions = all
25912024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/me/groups versions = all
25922024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/me/ssh-keys versions = all
25932024-12-13T21:18:25.154ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/me/ssh-keys versions = all
25942024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
25952024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
25962024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/metrics/{metric_name} versions = all
25972024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/network-interfaces versions = all
25982024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/network-interfaces versions = all
25992024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/network-interfaces/{interface} versions = all
26002024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/network-interfaces/{interface} versions = all
26012024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/network-interfaces/{interface} versions = all
26022024-12-13T21:18:25.155ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/ping versions = all
26032024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/policy versions = all
26042024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/policy versions = all
26052024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/projects versions = all
26062024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/projects versions = all
26072024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/projects/{project} versions = all
26082024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/projects/{project} versions = all
26092024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/projects/{project} versions = all
26102024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/projects/{project}/policy versions = all
26112024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/projects/{project}/policy versions = all
26122024-12-13T21:18:25.156ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/snapshots versions = all
26132024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/snapshots versions = all
26142024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/snapshots/{snapshot} versions = all
26152024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/snapshots/{snapshot} versions = all
26162024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/disks versions = all
26172024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
26182024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/racks versions = all
26192024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
26202024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/sleds versions = all
26212024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/hardware/sleds versions = all
26222024-12-13T21:18:25.157ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
26232024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
26242024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
26252024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
26262024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
26272024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/switch-port versions = all
26282024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
26292024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
26302024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
26312024-12-13T21:18:25.158ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/switches versions = all
26322024-12-13T21:18:25.159ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
26332024-12-13T21:18:25.159ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/identity-providers versions = all
26342024-12-13T21:18:25.159ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/identity-providers/local/users versions = all
26352024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
26362024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
26372024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/identity-providers/saml versions = all
26382024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
26392024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/ip-pools versions = all
26402024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/ip-pools versions = all
26412024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
26422024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/ip-pools/{pool} versions = all
26432024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/ip-pools/{pool} versions = all
26442024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
26452024-12-13T21:18:25.164ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
26462024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
26472024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
26482024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
26492024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
26502024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
26512024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
26522024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/ip-pools-service versions = all
26532024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/ip-pools-service/ranges versions = all
26542024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
26552024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
26562024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/metrics/{metric_name} versions = all
26572024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/address-lot versions = all
26582024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/networking/address-lot versions = all
26592024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
26602024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
26612024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/allow-list versions = all
26622024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/networking/allow-list versions = all
26632024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/networking/bfd-disable versions = all
26642024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/networking/bfd-enable versions = all
26652024-12-13T21:18:25.165ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/bfd-status versions = all
26662024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/networking/bgp versions = all
26672024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/bgp versions = all
26682024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/networking/bgp versions = all
26692024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/bgp-announce-set versions = all
26702024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
26712024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
26722024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
26732024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/bgp-exported versions = all
26742024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/bgp-message-history versions = all
26752024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
26762024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/bgp-status versions = all
26772024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/loopback-address versions = all
26782024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/networking/loopback-address versions = all
26792024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
26802024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
26812024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/switch-port-settings versions = all
26822024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/networking/switch-port-settings versions = all
26832024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
26842024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/policy versions = all
26852024-12-13T21:18:25.166ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/policy versions = all
26862024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/roles versions = all
26872024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/roles/{role_name} versions = all
26882024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/silo-quotas versions = all
26892024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/silos versions = all
26902024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/silos versions = all
26912024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/system/silos/{silo} versions = all
26922024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/silos/{silo} versions = all
26932024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
26942024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/silos/{silo}/policy versions = all
26952024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/silos/{silo}/policy versions = all
26962024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/silos/{silo}/quotas versions = all
26972024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
26982024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/system/timeseries/query versions = all
26992024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/timeseries/schemas versions = all
27002024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/system/update/repository versions = all
27012024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/update/repository/{system_version} versions = all
27022024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/users versions = all
27032024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/users/{user_id} versions = all
27042024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/users-builtin versions = all
27052024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/users-builtin/{user} versions = all
27062024-12-13T21:18:25.167ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/utilization/silos versions = all
27072024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/system/utilization/silos/{silo} versions = all
27082024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/timeseries/query versions = all
27092024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/users versions = all
27102024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/utilization versions = all
27112024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpc-firewall-rules versions = all
27122024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/vpc-firewall-rules versions = all
27132024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpc-router-routes versions = all
27142024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/vpc-router-routes versions = all
27152024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
27162024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpc-router-routes/{route} versions = all
27172024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/vpc-router-routes/{route} versions = all
27182024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpc-routers versions = all
27192024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/vpc-routers versions = all
27202024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/vpc-routers/{router} versions = all
27212024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpc-routers/{router} versions = all
27222024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/vpc-routers/{router} versions = all
27232024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpc-subnets versions = all
27242024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/vpc-subnets versions = all
27252024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
27262024-12-13T21:18:25.168ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpc-subnets/{subnet} versions = all
27272024-12-13T21:18:25.169ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
27282024-12-13T21:18:25.169ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
27292024-12-13T21:18:25.169ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpcs versions = all
27302024-12-13T21:18:25.169ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = POST path = /v1/vpcs versions = all
27312024-12-13T21:18:25.169ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = DELETE path = /v1/vpcs/{vpc} versions = all
27322024-12-13T21:18:25.169ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = GET path = /v1/vpcs/{vpc} versions = all
27332024-12-13T21:18:25.169ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:39489 method = PUT path = /v1/vpcs/{vpc} versions = all
27342024-12-13T21:18:25.169ZINFOtest_mgs_metrics (dropshot_external): listening local_addr = 127.0.0.1:39489
27352024-12-13T21:18:25.169ZDEBGtest_mgs_metrics (dropshot_external): successfully registered DTrace USDT probes local_addr = 127.0.0.1:39489
27362024-12-13T21:18:25.200ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = / versions = all
27372024-12-13T21:18:25.200ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /access versions = all
27382024-12-13T21:18:25.200ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /assets/{path} versions = all
27392024-12-13T21:18:25.200ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /device/auth versions = all
27402024-12-13T21:18:25.200ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /device/confirm versions = all
27412024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /device/success versions = all
27422024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /device/token versions = all
27432024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /device/verify versions = all
27442024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /experimental/v1/probes versions = all
27452024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /experimental/v1/probes versions = all
27462024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /experimental/v1/probes/{probe} versions = all
27472024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /experimental/v1/probes/{probe} versions = all
27482024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /images versions = all
27492024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /login versions = all
27502024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /login/{silo_name}/local versions = all
27512024-12-13T21:18:25.201ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
27522024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
27532024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
27542024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /lookup/{path} versions = all
27552024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /projects/{path} versions = all
27562024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /projects-new versions = all
27572024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /settings/{path} versions = all
27582024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /system/{path} versions = all
27592024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /utilization versions = all
27602024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/certificates versions = all
27612024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/certificates versions = all
27622024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/certificates/{certificate} versions = all
27632024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/certificates/{certificate} versions = all
27642024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/disks versions = all
27652024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/disks versions = all
27662024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/disks/{disk} versions = all
27672024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/disks/{disk} versions = all
27682024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/disks/{disk}/bulk-write versions = all
27692024-12-13T21:18:25.202ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
27702024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
27712024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/disks/{disk}/finalize versions = all
27722024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
27732024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/floating-ips versions = all
27742024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/floating-ips versions = all
27752024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
27762024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/floating-ips/{floating_ip} versions = all
27772024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
27782024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
27792024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
27802024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/groups versions = all
27812024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/groups/{group_id} versions = all
27822024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/images versions = all
27832024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/images versions = all
27842024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/images/{image} versions = all
27852024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/images/{image} versions = all
27862024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/images/{image}/demote versions = all
27872024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/images/{image}/promote versions = all
27882024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/instances versions = all
27892024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/instances versions = all
27902024-12-13T21:18:25.203ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/instances/{instance} versions = all
27912024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/instances/{instance} versions = all
27922024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/instances/{instance} versions = all
27932024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/instances/{instance}/disks versions = all
27942024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/instances/{instance}/disks/attach versions = all
27952024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/instances/{instance}/disks/detach versions = all
27962024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/instances/{instance}/external-ips versions = all
27972024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27982024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27992024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/instances/{instance}/reboot versions = all
28002024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/instances/{instance}/serial-console versions = all
28012024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
28022024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
28032024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/instances/{instance}/start versions = all
28042024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/instances/{instance}/stop versions = all
28052024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/internet-gateway-ip-addresses versions = all
28062024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/internet-gateway-ip-addresses versions = all
28072024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
28082024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/internet-gateway-ip-pools versions = all
28092024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/internet-gateway-ip-pools versions = all
28102024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
28112024-12-13T21:18:25.204ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/internet-gateways versions = all
28122024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/internet-gateways versions = all
28132024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
28142024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/internet-gateways/{gateway} versions = all
28152024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/ip-pools versions = all
28162024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/ip-pools/{pool} versions = all
28172024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/login/{silo_name}/local versions = all
28182024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/logout versions = all
28192024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/me versions = all
28202024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/me/groups versions = all
28212024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/me/ssh-keys versions = all
28222024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/me/ssh-keys versions = all
28232024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
28242024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
28252024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/metrics/{metric_name} versions = all
28262024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/network-interfaces versions = all
28272024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/network-interfaces versions = all
28282024-12-13T21:18:25.205ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/network-interfaces/{interface} versions = all
28292024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/network-interfaces/{interface} versions = all
28302024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/network-interfaces/{interface} versions = all
28312024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/ping versions = all
28322024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/policy versions = all
28332024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/policy versions = all
28342024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/projects versions = all
28352024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/projects versions = all
28362024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/projects/{project} versions = all
28372024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/projects/{project} versions = all
28382024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/projects/{project} versions = all
28392024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/projects/{project}/policy versions = all
28402024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/projects/{project}/policy versions = all
28412024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/snapshots versions = all
28422024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/snapshots versions = all
28432024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/snapshots/{snapshot} versions = all
28442024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/snapshots/{snapshot} versions = all
28452024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/disks versions = all
28462024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
28472024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/racks versions = all
28482024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
28492024-12-13T21:18:25.206ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/sleds versions = all
28502024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/hardware/sleds versions = all
28512024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
28522024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
28532024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
28542024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
28552024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
28562024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/switch-port versions = all
28572024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
28582024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
28592024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
28602024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/switches versions = all
28612024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
28622024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/identity-providers versions = all
28632024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/identity-providers/local/users versions = all
28642024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
28652024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
28662024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/identity-providers/saml versions = all
28672024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
28682024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/ip-pools versions = all
28692024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/ip-pools versions = all
28702024-12-13T21:18:25.207ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
28712024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/ip-pools/{pool} versions = all
28722024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/ip-pools/{pool} versions = all
28732024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
28742024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
28752024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
28762024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
28772024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
28782024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28792024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28802024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
28812024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/ip-pools-service versions = all
28822024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/ip-pools-service/ranges versions = all
28832024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
28842024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
28852024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/metrics/{metric_name} versions = all
28862024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/address-lot versions = all
28872024-12-13T21:18:25.208ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/networking/address-lot versions = all
28882024-12-13T21:18:25.209ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
28892024-12-13T21:18:25.209ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
28902024-12-13T21:18:25.209ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/allow-list versions = all
28912024-12-13T21:18:25.209ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/networking/allow-list versions = all
28922024-12-13T21:18:25.209ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/networking/bfd-disable versions = all
28932024-12-13T21:18:25.209ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/networking/bfd-enable versions = all
28942024-12-13T21:18:25.209ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/bfd-status versions = all
28952024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/networking/bgp versions = all
28962024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/bgp versions = all
28972024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/networking/bgp versions = all
28982024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/bgp-announce-set versions = all
28992024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
29002024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
29012024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
29022024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/bgp-exported versions = all
29032024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/bgp-message-history versions = all
29042024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
29052024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/bgp-status versions = all
29062024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/loopback-address versions = all
29072024-12-13T21:18:25.210ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/networking/loopback-address versions = all
29082024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
29092024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
29102024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/switch-port-settings versions = all
29112024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/networking/switch-port-settings versions = all
29122024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
29132024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/policy versions = all
29142024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/policy versions = all
29152024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/roles versions = all
29162024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/roles/{role_name} versions = all
29172024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/silo-quotas versions = all
29182024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/silos versions = all
29192024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/silos versions = all
29202024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/system/silos/{silo} versions = all
29212024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/silos/{silo} versions = all
29222024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
29232024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/silos/{silo}/policy versions = all
29242024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/silos/{silo}/policy versions = all
29252024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/silos/{silo}/quotas versions = all
29262024-12-13T21:18:25.211ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
29272024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/system/timeseries/query versions = all
29282024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/timeseries/schemas versions = all
29292024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/system/update/repository versions = all
29302024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/update/repository/{system_version} versions = all
29312024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/users versions = all
29322024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/users/{user_id} versions = all
29332024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/users-builtin versions = all
29342024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/users-builtin/{user} versions = all
29352024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/utilization/silos versions = all
29362024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/system/utilization/silos/{silo} versions = all
29372024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/timeseries/query versions = all
29382024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/users versions = all
29392024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/utilization versions = all
29402024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpc-firewall-rules versions = all
29412024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/vpc-firewall-rules versions = all
29422024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpc-router-routes versions = all
29432024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/vpc-router-routes versions = all
29442024-12-13T21:18:25.212ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
29452024-12-13T21:18:25.213ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpc-router-routes/{route} versions = all
29462024-12-13T21:18:25.213ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/vpc-router-routes/{route} versions = all
29472024-12-13T21:18:25.213ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpc-routers versions = all
29482024-12-13T21:18:25.213ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/vpc-routers versions = all
29492024-12-13T21:18:25.213ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/vpc-routers/{router} versions = all
29502024-12-13T21:18:25.213ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpc-routers/{router} versions = all
29512024-12-13T21:18:25.213ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/vpc-routers/{router} versions = all
29522024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpc-subnets versions = all
29532024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/vpc-subnets versions = all
29542024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
29552024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpc-subnets/{subnet} versions = all
29562024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
29572024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
29582024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpcs versions = all
29592024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = POST path = /v1/vpcs versions = all
29602024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = DELETE path = /v1/vpcs/{vpc} versions = all
29612024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = GET path = /v1/vpcs/{vpc} versions = all
29622024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:37467 method = PUT path = /v1/vpcs/{vpc} versions = all
29632024-12-13T21:18:25.217ZINFOtest_mgs_metrics (dropshot_external_techport): listening local_addr = 127.0.0.1:37467
29642024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = 127.0.0.1:37467
29652024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (nexus-producer-server): registered DTrace probes
29662024-12-13T21:18:25.217ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = 127.0.0.1:41365 method = GET path = /{producer_id} versions = all
29672024-12-13T21:18:25.217ZINFOtest_mgs_metrics (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = 127.0.0.1:41365
29682024-12-13T21:18:25.218ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = 127.0.0.1:41365
29692024-12-13T21:18:25.218ZDEBGtest_mgs_metrics (nexus-producer-server): Requested any available port, Dropshot server has been bound to 127.0.0.1:41365
29702024-12-13T21:18:25.218ZDEBGtest_mgs_metrics (nexus-producer-server): Nexus IP provided explicitly, will use it addr = 127.0.0.1:44649
29712024-12-13T21:18:25.218ZDEBGtest_mgs_metrics (nexus-producer-server): starting producer registration task
29722024-12-13T21:18:25.218ZINFOtest_mgs_metrics (nexus-producer-server): starting oximeter metric producer server address = 127.0.0.1:41365 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = 97039868-0386-49ec-9259-75c36edb1ac3
29732024-12-13T21:18:25.218ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29742024-12-13T21:18:25.218ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29752024-12-13T21:18:25.218ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29762024-12-13T21:18:25.218ZDEBGtest_mgs_metrics (producer-registration-task): registering / renewing oximeter producer lease with Nexus
29772024-12-13T21:18:25.218ZDEBGtest_mgs_metrics (producer-registration-task): using nexus address for registration addr = 127.0.0.1:44649
29782024-12-13T21:18:25.295ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:44649/metrics/producers
29792024-12-13T21:18:25.307ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:44649 remote_addr = 127.0.0.1:43006
29802024-12-13T21:18:25.307ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43006 req_id = e242ecdb-0b8c-4408-822b-60fdc0bb26c3 uri = /metrics/producers
29812024-12-13T21:18:25.307ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43006 req_id = e242ecdb-0b8c-4408-822b-60fdc0bb26c3 resource = Database uri = /metrics/producers
29822024-12-13T21:18:25.307ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43006 req_id = e242ecdb-0b8c-4408-822b-60fdc0bb26c3 roles = RoleSet { roles: {} } uri = /metrics/producers
29832024-12-13T21:18:25.307ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43006 req_id = e242ecdb-0b8c-4408-822b-60fdc0bb26c3 resource = Database result = Ok(()) uri = /metrics/producers
29842024-12-13T21:18:25.307ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
29852024-12-13T21:18:25.307ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29862024-12-13T21:18:25.307ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29872024-12-13T21:18:25.307ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29882024-12-13T21:18:25.308ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29892024-12-13T21:18:25.323ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29902024-12-13T21:18:25.328ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
29912024-12-13T21:18:25.328ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
29922024-12-13T21:18:25.328ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29932024-12-13T21:18:25.328ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29942024-12-13T21:18:25.328ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29952024-12-13T21:18:25.329ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29962024-12-13T21:18:25.344ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29972024-12-13T21:18:25.349ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
29982024-12-13T21:18:25.368ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29992024-12-13T21:18:25.368ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
30002024-12-13T21:18:25.368ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
30012024-12-13T21:18:25.481ZDEBGtest_mgs_metrics (init_with_steps): Running step start_oximeter
30022024-12-13T21:18:25.482ZDEBGtest_mgs_metrics: Starting Oximeter
30032024-12-13T21:18:25.482ZDEBGtest_mgs_metrics (oximeter): registered DTrace probes
30042024-12-13T21:18:25.482ZINFOtest_mgs_metrics (oximeter): starting oximeter server file = oximeter/collector/src/lib.rs:236
30052024-12-13T21:18:25.482ZDEBGtest_mgs_metrics (oximeter): creating ClickHouse client
30062024-12-13T21:18:25.482ZDEBGtest_mgs_metrics (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
30072024-12-13T21:18:25.482ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = SELECT MAX(value) AS max_version FROM oximeter.version;
30082024-12-13T21:18:25.487ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 180498 local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43006 req_id = e242ecdb-0b8c-4408-822b-60fdc0bb26c3 response_code = 503 uri = /metrics/producers
30092024-12-13T21:18:25.487ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:44649/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "e242ecdb-0b8c-4408-822b-60fdc0bb26c3", "content-length": "133", "date": "Fri, 13 Dec 2024 21:18:24 GMT"} })
30102024-12-13T21:18:25.487ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 254.190135ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"e242ecdb-0b8c-4408-822b-60fdc0bb26c3\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 13 Dec 2024 21:18:24 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"e242ecdb-0b8c-4408-822b-60fdc0bb26c3\\" }" file = oximeter/producer/src/lib.rs:420
30112024-12-13T21:18:25.496ZWARNtest_mgs_metrics (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:823 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
30122024-12-13T21:18:25.496ZDEBGtest_mgs_metrics (oximeter-agent): oximeter database does not exist, creating collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
30132024-12-13T21:18:25.496ZINFOtest_mgs_metrics (clickhouse-client): reading db version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:732 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
30142024-12-13T21:18:25.496ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = SELECT MAX(value) AS max_version FROM oximeter.version;
30152024-12-13T21:18:25.498ZWARNtest_mgs_metrics (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:823 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
30162024-12-13T21:18:25.498ZINFOtest_mgs_metrics (clickhouse-client): read oximeter database version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:736 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 version = 0
30172024-12-13T21:18:25.498ZINFOtest_mgs_metrics (clickhouse-client): wiping and re-initializing oximeter schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:741 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
30182024-12-13T21:18:25.498ZDEBGtest_mgs_metrics (clickhouse-client): wiping ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
30192024-12-13T21:18:25.498ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = DROP DATABASE IF EXISTS oximeter
30202024-12-13T21:18:25.498ZDEBGtest_mgs_metrics (clickhouse-client): initializing ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
30212024-12-13T21:18:25.498ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = CREATE DATABASE IF NOT EXISTS oximeter
30222024-12-13T21:18:25.508ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\n/* The version table contains metadata about the `oximeter` database */\nCREATE TABLE IF NOT EXISTS oximeter.version\n(\n value UInt64,\n timestamp DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (value, timestamp)
30232024-12-13T21:18:25.520ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\n/* The measurement tables contain all individual samples from each timeseries.\n *\n * Each table stores a single datum type, and otherwise contains nearly the same\n * structure. The primary sorting key is on the timeseries name, key, and then\n * timestamp, so that all timeseries from the same schema are grouped, followed\n * by all samples from the same timeseries.\n *\n * This reflects that one usually looks up the _key_ in one or more field table,\n * and then uses that to index quickly into the measurements tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Bool)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30242024-12-13T21:18:25.540ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30252024-12-13T21:18:25.564ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30262024-12-13T21:18:25.574ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30272024-12-13T21:18:25.589ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30282024-12-13T21:18:25.600ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30292024-12-13T21:18:25.613ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30302024-12-13T21:18:25.628ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30312024-12-13T21:18:25.644ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30322024-12-13T21:18:25.656ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30332024-12-13T21:18:25.671ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30342024-12-13T21:18:25.684ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(String)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30352024-12-13T21:18:25.698ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bytes\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Right now we can't unambiguously record a nullable byte array.\n * Arrays cannot be nested in `Nullable()` types, and encoding the array as\n * a string isn't palatable for a few reasons.\n * See: https://github.com/oxidecomputer/omicron/issues/4551 for more\n * details.\n */\n datum Array(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30362024-12-13T21:18:25.709ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativei64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30372024-12-13T21:18:25.724ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativeu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30382024-12-13T21:18:25.737ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30392024-12-13T21:18:25.783ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:44649/metrics/producers
30402024-12-13T21:18:25.784ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30412024-12-13T21:18:25.784ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:44649 remote_addr = 127.0.0.1:43008
30422024-12-13T21:18:25.787ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43008 req_id = e63f30dc-9790-4437-9490-bd03f714e67c uri = /metrics/producers
30432024-12-13T21:18:25.787ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43008 req_id = e63f30dc-9790-4437-9490-bd03f714e67c resource = Database uri = /metrics/producers
30442024-12-13T21:18:25.788ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43008 req_id = e63f30dc-9790-4437-9490-bd03f714e67c roles = RoleSet { roles: {} } uri = /metrics/producers
30452024-12-13T21:18:25.789ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43008 req_id = e63f30dc-9790-4437-9490-bd03f714e67c resource = Database result = Ok(()) uri = /metrics/producers
30462024-12-13T21:18:25.805ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Array types cannot be Nullable, see\n * https://clickhouse.com/docs/en/sql-reference/data-types/nullable\n * for more details.\n *\n * This means we need to use empty arrays to indicate a missing value. This\n * is unfortunate, and at this point relies on the fact that an\n * `oximeter::Histogram` cannot have zero bins. If that changes, we'll need\n * to figure out another way to represent missing samples here.\n */\n bins Array(Int8),\n counts Array(UInt64),\n min Int8,\n max Int8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30472024-12-13T21:18:25.810ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 95 eb 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:65098 req_id = e630c3ae-f43f-470b-aeb9-382652ece683
30482024-12-13T21:18:25.810ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38379,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65098 req_id = e630c3ae-f43f-470b-aeb9-382652ece683
30492024-12-13T21:18:25.812ZDEBGtest_mgs_metrics (store): query key key = _crucible-pantry._tcp
30502024-12-13T21:18:25.812ZDEBGtest_mgs_metrics (store): query key key = e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host
30512024-12-13T21:18:25.812ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65098 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 33815, target: Name("e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host.control-plane.oxide.internal") })) }] req_id = e630c3ae-f43f-470b-aeb9-382652ece683
30522024-12-13T21:18:25.817ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 08 38 01 00 00 01 00 00 00 00 00 01 24 65 37 36 .8..........$e76\n0010: 65 31 64 30 63 2d 33 65 62 30 2d 34 33 39 30 2d e1d0c-3eb0-4390-\n0020: 39 31 31 64 2d 66 64 66 65 31 39 38 35 37 65 64 911d-fdfe19857ed\n0030: 36 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 6.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:61996 req_id = 2a05dbe4-10e8-4b6e-9de3-8e6ed127f467
30532024-12-13T21:18:25.817ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2104,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 101,\n 55,\n 54,\n 101,\n 49,\n 100,\n 48,\n 99,\n 45,\n 51,\n 101,\n 98,\n 48,\n 45,\n 52,\n 51,\n 57,\n 48,\n 45,\n 57,\n 49,\n 49,\n 100,\n 45,\n 102,\n 100,\n 102,\n 101,\n 49,\n 57,\n 56,\n 53,\n 55,\n 101,\n 100,\n 54,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61996 req_id = 2a05dbe4-10e8-4b6e-9de3-8e6ed127f467
30542024-12-13T21:18:25.817ZDEBGtest_mgs_metrics (store): query key key = e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host
30552024-12-13T21:18:25.817ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61996 query = LowerQuery { name: LowerName(Name("e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host.control-plane.oxide.internal.")), original: Query { name: Name("e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("e76e1d0c-3eb0-4390-911d-fdfe19857ed6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 2a05dbe4-10e8-4b6e-9de3-8e6ed127f467
30562024-12-13T21:18:25.907ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 120456 local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43008 req_id = e63f30dc-9790-4437-9490-bd03f714e67c response_code = 503 uri = /metrics/producers
30572024-12-13T21:18:25.909ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt8),\n counts Array(UInt64),\n min UInt8,\n max UInt8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30582024-12-13T21:18:25.909ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:33815 remote_addr = [::1]:53820 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30592024-12-13T21:18:25.910ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:44649/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "e63f30dc-9790-4437-9490-bd03f714e67c", "content-length": "133", "date": "Fri, 13 Dec 2024 21:18:24 GMT"} })
30602024-12-13T21:18:25.913ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 596.042253ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"e63f30dc-9790-4437-9490-bd03f714e67c\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 13 Dec 2024 21:18:24 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"e63f30dc-9790-4437-9490-bd03f714e67c\\" }" file = oximeter/producer/src/lib.rs:420
30612024-12-13T21:18:25.919ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:33815 method = GET remote_addr = [::1]:53820 req_id = 4271b0a9-fa3c-4e2c-a649-b1bf884c19c0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30622024-12-13T21:18:25.921ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 717 local_addr = [::1]:33815 method = GET remote_addr = [::1]:53820 req_id = 4271b0a9-fa3c-4e2c-a649-b1bf884c19c0 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30632024-12-13T21:18:26.436ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int16),\n counts Array(UInt64),\n min Int16,\n max Int16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30642024-12-13T21:18:26.436ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:33815 remote_addr = [::1]:53822 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30652024-12-13T21:18:26.436ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:33815 remote_addr = [::1]:53824 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30662024-12-13T21:18:26.436ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:33815 remote_addr = [::1]:53826 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30672024-12-13T21:18:26.436ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:33815 remote_addr = [::1]:53828 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30682024-12-13T21:18:26.436ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:33815 remote_addr = [::1]:53830 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30692024-12-13T21:18:26.438ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:33815 remote_addr = [::1]:53832 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30702024-12-13T21:18:26.444ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:33815 remote_addr = [::1]:53834 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30712024-12-13T21:18:26.444ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:33815 method = GET remote_addr = [::1]:53822 req_id = b0817689-ade2-4f41-acde-d9bddb69a884 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30722024-12-13T21:18:26.444ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:33815 method = GET remote_addr = [::1]:53824 req_id = 9d4fc2aa-0ddb-48c3-8850-dff6675670ad sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30732024-12-13T21:18:26.444ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:33815 method = GET remote_addr = [::1]:53826 req_id = 21db2288-4f3b-4677-a6e5-c2dd7ae83794 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30742024-12-13T21:18:26.444ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:33815 method = GET remote_addr = [::1]:53828 req_id = 06fca6b6-9b31-492a-877e-e3d5c529c679 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30752024-12-13T21:18:26.444ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:33815 method = GET remote_addr = [::1]:53830 req_id = 5a0f3ed6-c769-4603-baf3-49b29ad5a0e5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30762024-12-13T21:18:26.444ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:33815 method = GET remote_addr = [::1]:53832 req_id = 7c07dc4a-3985-4582-80a6-baefa32ae62d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30772024-12-13T21:18:26.444ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:33815 method = GET remote_addr = [::1]:53834 req_id = ac2aad13-ca2f-4a9d-a2cf-223b8e537844 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30782024-12-13T21:18:26.444ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1417 local_addr = [::1]:33815 method = GET remote_addr = [::1]:53822 req_id = b0817689-ade2-4f41-acde-d9bddb69a884 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30792024-12-13T21:18:26.445ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1357 local_addr = [::1]:33815 method = GET remote_addr = [::1]:53824 req_id = 9d4fc2aa-0ddb-48c3-8850-dff6675670ad response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30802024-12-13T21:18:26.445ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1299 local_addr = [::1]:33815 method = GET remote_addr = [::1]:53826 req_id = 21db2288-4f3b-4677-a6e5-c2dd7ae83794 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30812024-12-13T21:18:26.445ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1239 local_addr = [::1]:33815 method = GET remote_addr = [::1]:53828 req_id = 06fca6b6-9b31-492a-877e-e3d5c529c679 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30822024-12-13T21:18:26.445ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1194 local_addr = [::1]:33815 method = GET remote_addr = [::1]:53830 req_id = 5a0f3ed6-c769-4603-baf3-49b29ad5a0e5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30832024-12-13T21:18:26.445ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1133 local_addr = [::1]:33815 method = GET remote_addr = [::1]:53832 req_id = 7c07dc4a-3985-4582-80a6-baefa32ae62d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30842024-12-13T21:18:26.445ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1078 local_addr = [::1]:33815 method = GET remote_addr = [::1]:53834 req_id = ac2aad13-ca2f-4a9d-a2cf-223b8e537844 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30852024-12-13T21:18:26.472ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt16),\n counts Array(UInt64),\n min UInt16,\n max UInt16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30862024-12-13T21:18:26.506ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int32),\n counts Array(UInt64),\n min Int32,\n max Int32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30872024-12-13T21:18:26.575ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:44649/metrics/producers
30882024-12-13T21:18:26.579ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt32),\n counts Array(UInt64),\n min UInt32,\n max UInt32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30892024-12-13T21:18:26.580ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:44649 remote_addr = 127.0.0.1:43010
30902024-12-13T21:18:26.581ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43010 req_id = 4e19e06d-5641-42d8-97ff-3262ac6f0dd3 uri = /metrics/producers
30912024-12-13T21:18:26.588ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43010 req_id = 4e19e06d-5641-42d8-97ff-3262ac6f0dd3 resource = Database uri = /metrics/producers
30922024-12-13T21:18:26.588ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43010 req_id = 4e19e06d-5641-42d8-97ff-3262ac6f0dd3 roles = RoleSet { roles: {} } uri = /metrics/producers
30932024-12-13T21:18:26.588ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43010 req_id = 4e19e06d-5641-42d8-97ff-3262ac6f0dd3 resource = Database result = Ok(()) uri = /metrics/producers
30942024-12-13T21:18:26.590ZINFOtest_mgs_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = edc023da-7eec-4453-b023-59ff762cb9af
30952024-12-13T21:18:26.590ZDEBGtest_mgs_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = edc023da-7eec-4453-b023-59ff762cb9af time_started = 2024-12-13 21:18:24.437837 UTC
30962024-12-13T21:18:26.590ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.681587143s iteration = 4 status = {"collection_id":"edc023da-7eec-4453-b023-59ff762cb9af","time_done":"2024-12-13 21:18:24.994728 UTC","time_started":"2024-12-13 21:18:24.437837 UTC"}
30972024-12-13T21:18:26.590ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
30982024-12-13T21:18:26.590ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 33.43\u{b5}s iteration = 2 status = {"error":"task disabled"}
30992024-12-13T21:18:26.604ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 22162 local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43010 req_id = 4e19e06d-5641-42d8-97ff-3262ac6f0dd3 response_code = 503 uri = /metrics/producers
31002024-12-13T21:18:26.604ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int64),\n counts Array(UInt64),\n min Int64,\n max Int64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31012024-12-13T21:18:26.605ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:44649/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "4e19e06d-5641-42d8-97ff-3262ac6f0dd3", "content-length": "133", "date": "Fri, 13 Dec 2024 21:18:25 GMT"} })
31022024-12-13T21:18:26.606ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.164717015s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"4e19e06d-5641-42d8-97ff-3262ac6f0dd3\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 13 Dec 2024 21:18:25 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"4e19e06d-5641-42d8-97ff-3262ac6f0dd3\\" }" file = oximeter/producer/src/lib.rs:420
31032024-12-13T21:18:26.627ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt64),\n counts Array(UInt64),\n min UInt64,\n max UInt64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31042024-12-13T21:18:26.660ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float32),\n counts Array(UInt64),\n min Float32,\n max Float32,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31052024-12-13T21:18:26.678ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float64),\n counts Array(UInt64),\n min Float64,\n max Float64,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31062024-12-13T21:18:26.704ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\n/* The field tables store named dimensions of each timeseries.\n *\n * As with the measurement tables, there is one field table for each field data\n * type. Fields are deduplicated by using the "replacing merge tree", though\n * this behavior **must not** be relied upon for query correctness.\n *\n * The index for the fields differs from the measurements, however. Rows are\n * sorted by timeseries name, then field name, field value, and finally\n * timeseries key. This reflects the most common pattern for looking them up:\n * by field name and possibly value, within a timeseries. The resulting keys are\n * usually then used to look up measurements.\n *\n * NOTE: We may want to consider a secondary index on these tables, sorting by\n * timeseries name and then key, since it would improve lookups where one\n * already has the key. Realistically though, these tables are quite small and\n * so performance benefits will be low in absolute terms.\n *\n * TTL: We use a materialized column to expire old field table records. This\n * column is generated automatically by the database whenever a new row is\n * inserted. It cannot be inserted directly, nor is it returned in a `SELECT *`\n * query. Since these tables are `ReplacingMergeTree`s, that means the last\n * record will remain during a deduplication, which will have the last\n * timestamp. ClickHouse will then expire old data for us, similar to the\n * measurement tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.fields_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Bool,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31072024-12-13T21:18:26.719ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31082024-12-13T21:18:26.733ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31092024-12-13T21:18:26.749ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31102024-12-13T21:18:26.769ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31112024-12-13T21:18:26.790ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31122024-12-13T21:18:26.808ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31132024-12-13T21:18:26.829ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31142024-12-13T21:18:26.848ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31152024-12-13T21:18:26.863ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_ipaddr\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value IPv6,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31162024-12-13T21:18:26.891ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value String,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31172024-12-13T21:18:26.906ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_uuid\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UUID,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31182024-12-13T21:18:26.925ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = \n\n/* The timeseries schema table stores the extracted schema for the samples\n * oximeter collects.\n */\nCREATE TABLE IF NOT EXISTS oximeter.timeseries_schema\n(\n timeseries_name String,\n fields Nested(\n name String,\n type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'IpAddr' = 3,\n 'String' = 4,\n 'Uuid' = 6,\n 'I8' = 7,\n 'U8' = 8,\n 'I16' = 9,\n 'U16' = 10,\n 'I32' = 11,\n 'U32' = 12,\n 'U64' = 13\n ),\n source Enum(\n 'Target' = 1,\n 'Metric' = 2\n )\n ),\n datum_type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'F64' = 3,\n 'String' = 4,\n 'Bytes' = 5,\n 'CumulativeI64' = 6,\n 'CumulativeF64' = 7,\n 'HistogramI64' = 8,\n 'HistogramF64' = 9,\n 'I8' = 10,\n 'U8' = 11,\n 'I16' = 12,\n 'U16' = 13,\n 'I32' = 14,\n 'U32' = 15,\n 'U64' = 16,\n 'F32' = 17,\n 'CumulativeU64' = 18,\n 'CumulativeF32' = 19,\n 'HistogramI8' = 20,\n 'HistogramU8' = 21,\n 'HistogramI16' = 22,\n 'HistogramU16' = 23,\n 'HistogramI32' = 24,\n 'HistogramU32' = 25,\n 'HistogramU64' = 26,\n 'HistogramF32' = 27\n ),\n created DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, fields.name)
31192024-12-13T21:18:26.941ZINFOtest_mgs_metrics (clickhouse-client): inserting current version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:764 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 version = 13
31202024-12-13T21:18:26.943ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219 sql = INSERT INTO oximeter.version (*) VALUES (13, now());
31212024-12-13T21:18:26.993ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45595 method = GET path = /info versions = all
31222024-12-13T21:18:26.994ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45595 method = GET path = /producers versions = all
31232024-12-13T21:18:26.994ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45595 method = DELETE path = /producers/{producer_id} versions = all
31242024-12-13T21:18:26.994ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45595 method = GET path = /producers/{producer_id} versions = all
31252024-12-13T21:18:26.994ZINFOtest_mgs_metrics (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [::1]:45595
31262024-12-13T21:18:26.994ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:45595
31272024-12-13T21:18:26.994ZDEBGtest_mgs_metrics (oximeter): registered USDT probes
31282024-12-13T21:18:26.994ZDEBGtest_mgs_metrics (oximeter): contacting nexus
31292024-12-13T21:18:27.366ZDEBGtest_mgs_metrics (oximeter): client request body = Some(Body) method = POST uri = http://127.0.0.1:44649/metrics/collectors
31302024-12-13T21:18:27.374ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:44649 remote_addr = 127.0.0.1:43012
31312024-12-13T21:18:27.375ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43012 req_id = 58c61484-545c-4972-87ac-5d6b9015e322 uri = /metrics/collectors
31322024-12-13T21:18:27.376ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43012 req_id = 58c61484-545c-4972-87ac-5d6b9015e322 resource = Database uri = /metrics/collectors
31332024-12-13T21:18:27.376ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43012 req_id = 58c61484-545c-4972-87ac-5d6b9015e322 roles = RoleSet { roles: {} } uri = /metrics/collectors
31342024-12-13T21:18:27.377ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43012 req_id = 58c61484-545c-4972-87ac-5d6b9015e322 resource = Database result = Ok(()) uri = /metrics/collectors
31352024-12-13T21:18:27.422ZINFOtest_mgs_metrics (nexus): registered new oximeter metric collection server address = [::1]:45595 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31362024-12-13T21:18:27.422ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 46678 local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43012 req_id = 58c61484-545c-4972-87ac-5d6b9015e322 response_code = 204 uri = /metrics/collectors
31372024-12-13T21:18:27.424ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:44649/metrics/collectors", status: 204, headers: {"x-request-id": "58c61484-545c-4972-87ac-5d6b9015e322", "date": "Fri, 13 Dec 2024 21:18:27 GMT"} })
31382024-12-13T21:18:27.424ZINFOtest_mgs_metrics (oximeter): oximeter registered with nexus file = oximeter/collector/src/lib.rs:383 id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31392024-12-13T21:18:27.424ZDEBGtest_mgs_metrics (init_with_steps): Running step start_producer_server
31402024-12-13T21:18:27.424ZDEBGtest_mgs_metrics: Starting test metric Producer Server
31412024-12-13T21:18:27.444ZINFOtest_mgs_metrics (oximeter-agent): starting refresh list task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:471
31422024-12-13T21:18:27.538ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:44649 remote_addr = 127.0.0.1:43014
31432024-12-13T21:18:27.538ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:44649 remote_addr = 127.0.0.1:43016
31442024-12-13T21:18:27.539ZINFOtest_mgs_metrics (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:474
31452024-12-13T21:18:27.539ZDEBGtest_mgs_metrics (oximeter): client request body = None method = GET uri = http://127.0.0.1:44649/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31462024-12-13T21:18:27.541ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43014 req_id = 454534a3-16bf-4732-89a6-9d5524f79225 uri = /metrics/producers
31472024-12-13T21:18:27.541ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43016 req_id = 4cc0fb56-7a34-494d-afe9-d25a56350cdc uri = /metrics/producers
31482024-12-13T21:18:27.542ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = e7079cb0-39b1-43fc-8d45-08fe28757240 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31492024-12-13T21:18:27.542ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43014 req_id = 454534a3-16bf-4732-89a6-9d5524f79225 resource = Database uri = /metrics/producers
31502024-12-13T21:18:27.542ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43014 req_id = 454534a3-16bf-4732-89a6-9d5524f79225 roles = RoleSet { roles: {} } uri = /metrics/producers
31512024-12-13T21:18:27.545ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43014 req_id = 454534a3-16bf-4732-89a6-9d5524f79225 resource = Database result = Ok(()) uri = /metrics/producers
31522024-12-13T21:18:27.545ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43016 req_id = 4cc0fb56-7a34-494d-afe9-d25a56350cdc resource = Database uri = /metrics/producers
31532024-12-13T21:18:27.545ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43016 req_id = 4cc0fb56-7a34-494d-afe9-d25a56350cdc roles = RoleSet { roles: {} } uri = /metrics/producers
31542024-12-13T21:18:27.546ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43016 req_id = 4cc0fb56-7a34-494d-afe9-d25a56350cdc resource = Database result = Ok(()) uri = /metrics/producers
31552024-12-13T21:18:27.554ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = e7079cb0-39b1-43fc-8d45-08fe28757240 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31562024-12-13T21:18:27.557ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = e7079cb0-39b1-43fc-8d45-08fe28757240 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31572024-12-13T21:18:27.557ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = e7079cb0-39b1-43fc-8d45-08fe28757240 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31582024-12-13T21:18:27.586ZINFOtest_mgs_metrics (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31592024-12-13T21:18:27.587ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 45837 local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43016 req_id = 4cc0fb56-7a34-494d-afe9-d25a56350cdc response_code = 201 uri = /metrics/producers
31602024-12-13T21:18:27.590ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 48596 local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = e7079cb0-39b1-43fc-8d45-08fe28757240 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31612024-12-13T21:18:27.590ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:44649/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7079cb0-39b1-43fc-8d45-08fe28757240", "content-length": "293", "date": "Fri, 13 Dec 2024 21:18:27 GMT"} })
31622024-12-13T21:18:27.590ZDEBGtest_mgs_metrics (oximeter): client request body = None method = GET uri = http://127.0.0.1:44649/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI3Y2MyYmUwMS00YmNmLTRiNWYtOWUxYi1kODQxNjNkYjI5N2IifX0%3D
31632024-12-13T21:18:27.591ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = 0c14adaf-bf22-4b28-9678-3d1d9b39bcb3 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI3Y2MyYmUwMS00YmNmLTRiNWYtOWUxYi1kODQxNjNkYjI5N2IifX0%3D
31642024-12-13T21:18:27.591ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = 0c14adaf-bf22-4b28-9678-3d1d9b39bcb3 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI3Y2MyYmUwMS00YmNmLTRiNWYtOWUxYi1kODQxNjNkYjI5N2IifX0%3D
31652024-12-13T21:18:27.591ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = 0c14adaf-bf22-4b28-9678-3d1d9b39bcb3 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI3Y2MyYmUwMS00YmNmLTRiNWYtOWUxYi1kODQxNjNkYjI5N2IifX0%3D
31662024-12-13T21:18:27.594ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = 0c14adaf-bf22-4b28-9678-3d1d9b39bcb3 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI3Y2MyYmUwMS00YmNmLTRiNWYtOWUxYi1kODQxNjNkYjI5N2IifX0%3D
31672024-12-13T21:18:27.599ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 8774 local_addr = 127.0.0.1:44649 method = GET remote_addr = 127.0.0.1:43012 req_id = 0c14adaf-bf22-4b28-9678-3d1d9b39bcb3 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI3Y2MyYmUwMS00YmNmLTRiNWYtOWUxYi1kODQxNjNkYjI5N2IifX0%3D
31682024-12-13T21:18:27.600ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:44649/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI3Y2MyYmUwMS00YmNmLTRiNWYtOWUxYi1kODQxNjNkYjI5N2IifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c14adaf-bf22-4b28-9678-3d1d9b39bcb3", "content-length": "29", "date": "Fri, 13 Dec 2024 21:18:27 GMT"} })
31692024-12-13T21:18:27.600ZDEBGtest_mgs_metrics (oximeter-agent): registered new metric producer address = [::1]:44751 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31702024-12-13T21:18:27.644ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31712024-12-13T21:18:27.644ZINFOtest_mgs_metrics (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31722024-12-13T21:18:27.644ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 103451 local_addr = 127.0.0.1:44649 method = POST remote_addr = 127.0.0.1:43014 req_id = 454534a3-16bf-4732-89a6-9d5524f79225 response_code = 201 uri = /metrics/producers
31732024-12-13T21:18:27.646ZINFOtest_mgs_metrics (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:549 n_current_tasks = 1 n_pruned_tasks = 0
31742024-12-13T21:18:27.654ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31752024-12-13T21:18:27.654ZTRACtest_mgs_metrics (collection-task): forwarded explicit request to collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31762024-12-13T21:18:27.654ZDEBGtest_mgs_metrics (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31772024-12-13T21:18:27.656ZTRACtest_mgs_metrics (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31782024-12-13T21:18:27.657ZDEBGtest_mgs_metrics (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 2 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31792024-12-13T21:18:27.657ZTRACtest_mgs_metrics (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31802024-12-13T21:18:27.657ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31812024-12-13T21:18:27.657ZDEBGtest_mgs_metrics (results-sink): inserting 0 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31822024-12-13T21:18:27.657ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 0 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
31832024-12-13T21:18:27.657ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31842024-12-13T21:18:27.657ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31852024-12-13T21:18:27.657ZTRACtest_mgs_metrics (collection-task): forwarded explicit request to collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31862024-12-13T21:18:27.658ZDEBGtest_mgs_metrics (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31872024-12-13T21:18:27.658ZTRACtest_mgs_metrics (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31882024-12-13T21:18:27.658ZDEBGtest_mgs_metrics (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 2 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31892024-12-13T21:18:27.658ZTRACtest_mgs_metrics (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31902024-12-13T21:18:27.658ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 7cc2be01-4bcf-4b5f-9e1b-d84163db297b
31912024-12-13T21:18:27.658ZDEBGtest_mgs_metrics (results-sink): inserting 0 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31922024-12-13T21:18:27.658ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 0 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 7372ba8d-5bc9-4fd2-8b07-cdb5f331d219
31932024-12-13T21:18:27.658ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31942024-12-13T21:18:27.658ZINFOtest_mgs_metrics (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:39489/v1/system/timeseries/query
31952024-12-13T21:18:27.658ZINFOtest_mgs_metrics (dropshot_external): accepted connection local_addr = 127.0.0.1:39489 remote_addr = 127.0.0.1:47148
31962024-12-13T21:18:27.659ZTRACtest_mgs_metrics (dropshot_external): incoming request local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 uri = /v1/system/timeseries/query
31972024-12-13T21:18:27.660ZTRACtest_mgs_metrics (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 uri = /v1/system/timeseries/query
31982024-12-13T21:18:27.660ZDEBGtest_mgs_metrics (dropshot_external): looking up silo for user local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/timeseries/query
31992024-12-13T21:18:27.660ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32002024-12-13T21:18:27.660ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32012024-12-13T21:18:27.660ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32022024-12-13T21:18:27.668ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32032024-12-13T21:18:27.668ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32042024-12-13T21:18:27.668ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32052024-12-13T21:18:27.676ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
32062024-12-13T21:18:27.676ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32072024-12-13T21:18:27.677ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32082024-12-13T21:18:27.677ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32092024-12-13T21:18:27.677ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32102024-12-13T21:18:27.683ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32112024-12-13T21:18:27.683ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32122024-12-13T21:18:27.683ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32132024-12-13T21:18:27.683ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32142024-12-13T21:18:27.697ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32152024-12-13T21:18:27.704ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
32162024-12-13T21:18:27.706ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32172024-12-13T21:18:27.706ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32182024-12-13T21:18:27.706ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32192024-12-13T21:18:27.713ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
32202024-12-13T21:18:27.713ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32212024-12-13T21:18:27.713ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32222024-12-13T21:18:27.713ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32232024-12-13T21:18:27.713ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32242024-12-13T21:18:27.717ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32252024-12-13T21:18:27.720ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32262024-12-13T21:18:27.721ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32272024-12-13T21:18:27.722ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32282024-12-13T21:18:27.723ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32292024-12-13T21:18:27.726ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
32302024-12-13T21:18:27.726ZTRACtest_mgs_metrics (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 uri = /v1/system/timeseries/query
32312024-12-13T21:18:27.726ZTRACtest_mgs_metrics (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 resource = Fleet uri = /v1/system/timeseries/query
32322024-12-13T21:18:27.727ZTRACtest_mgs_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/timeseries/query
32332024-12-13T21:18:27.727ZTRACtest_mgs_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 resource = Database uri = /v1/system/timeseries/query
32342024-12-13T21:18:27.727ZDEBGtest_mgs_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 roles = RoleSet { roles: {} } uri = /v1/system/timeseries/query
32352024-12-13T21:18:27.727ZDEBGtest_mgs_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 resource = Database result = Ok(()) uri = /v1/system/timeseries/query
32362024-12-13T21:18:27.747ZDEBGtest_mgs_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/timeseries/query
32372024-12-13T21:18:27.753ZDEBGtest_mgs_metrics (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 resource = Fleet result = Ok(()) uri = /v1/system/timeseries/query
32382024-12-13T21:18:27.759ZDEBGtest_mgs_metrics (clickhouse-client): parsed OxQL query id = 272f9fcc-700b-45ef-85a6-81a18631e24b parsed_query = Query { parsed: Query { ops: [Basic(Get(TimeseriesName("hardware_component:temperature")))] }, end_time: 2024-12-13T21:18:27.754381361Z } query = get hardware_component:temperature query_id = 544fdb50-b4b3-49b3-9d86-8bb0ffcf069a
32392024-12-13T21:18:27.759ZDEBGtest_mgs_metrics (clickhouse-client): retrieving timeseries schema from database id = 272f9fcc-700b-45ef-85a6-81a18631e24b
32402024-12-13T21:18:27.759ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query id = 272f9fcc-700b-45ef-85a6-81a18631e24b sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
32412024-12-13T21:18:27.759ZTRACtest_mgs_metrics (clickhouse-client): no new timeseries schema in database id = 272f9fcc-700b-45ef-85a6-81a18631e24b
32422024-12-13T21:18:27.759ZINFOtest_mgs_metrics (dropshot_external): request completed error_message_external = Timeseries not found for: hardware_component:temperature error_message_internal = Timeseries not found for: hardware_component:temperature latency_us = 98695 local_addr = 127.0.0.1:39489 method = POST remote_addr = 127.0.0.1:47148 req_id = 6bb28704-c625-41ab-8066-0e1eab103ed7 response_code = 400 uri = /v1/system/timeseries/query
32432024-12-13T21:18:27.759ZINFOtest_mgs_metrics (external client test context): client received response status = 400