Max level shown:
12024-11-15T20:11:19.435ZDEBGtest_mgs_metrics (init_with_steps): Running step start_crdb
22024-11-15T20:11:19.435ZDEBGtest_mgs_metrics: Starting CRDB
32024-11-15T20:11:19.435ZINFOtest_mgs_metrics: cockroach temporary directory: /var/tmp/omicron_tmp/.tmphYxkpE
42024-11-15T20:11:19.435ZINFOtest_mgs_metrics: cockroach: copying from seed tarball (/var/tmp/omicron_tmp/crdb-base-build/01bc54cd6557f78626486ef45eaa015c5c96a6700d7760418aa2aaba8e57ab25.tar) to storage directory (/var/tmp/omicron_tmp/.tmphYxkpE/data)
52024-11-15T20:11:19.469ZINFOtest_mgs_metrics: cockroach command line: cockroach start-single-node --insecure --http-addr=:0 --store=path=/var/tmp/omicron_tmp/.tmphYxkpE/data,ballast-size=0 --listen-addr [::1]:0 --listening-url-file /var/tmp/omicron_tmp/.tmphYxkpE/listen-url
62024-11-15T20:11:19.469ZINFOtest_mgs_metrics: cockroach environment: BUILDOMAT_JOB_ID=01JCRN8WRZX5NQX1EFKV3DX7GP BUILDOMAT_TASK_ID=4 CARGO=/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/bin/cargo CARGO_HOME=/home/build/.cargo CARGO_INCREMENTAL=0 CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus CARGO_PKG_AUTHORS= CARGO_PKG_DESCRIPTION= CARGO_PKG_HOMEPAGE= CARGO_PKG_LICENSE=MPL-2.0 CARGO_PKG_LICENSE_FILE= CARGO_PKG_NAME=omicron-nexus CARGO_PKG_REPOSITORY= CARGO_PKG_RUST_VERSION= CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= CARGO_TERM_COLOR=always CI=true CRDB_SEED_TAR=/var/tmp/omicron_tmp/crdb-base-build/01bc54cd6557f78626486ef45eaa015c5c96a6700d7760418aa2aaba8e57ab25.tar GITHUB_BRANCH=dendrite-integration GITHUB_REF=refs/heads/dendrite-integration GITHUB_REPOSITORY=oxidecomputer/omicron GITHUB_SHA=a8f2400c1930fa7912ac43d6f6c340464360b20d GOTRACEBACK=crash HOME=/home/build LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-0605127ef2db4010/out:/work/oxidecomputer/omicron/target/debug/build/blake3-1e464eee91b26c77/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-70e33ff4b45d8dc0/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-6a6e82c218d03cb7/out/build:/work/oxidecomputer/omicron/target/debug/build/libz-sys-33d79b7b3a6a5533/out/lib:/work/oxidecomputer/omicron/target/debug/build/ring-84313340968b144b/out:/work/oxidecomputer/omicron/target/debug/build/ring-ae424a22a6f8daa6/out:/work/oxidecomputer/omicron/target/debug/build/tofino-b9675a1b9b2f961b/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib LOGNAME=build NEXTEST=1 NEXTEST_BIN_EXE_nexus=/work/oxidecomputer/omicron/target/debug/nexus NEXTEST_BIN_EXE_schema-updater=/work/oxidecomputer/omicron/target/debug/schema-updater NEXTEST_EXECUTION_MODE=process-per-test NEXTEST_LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-0605127ef2db4010/out:/work/oxidecomputer/omicron/target/debug/build/blake3-1e464eee91b26c77/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-70e33ff4b45d8dc0/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-6a6e82c218d03cb7/out/build:/work/oxidecomputer/omicron/target/debug/build/libz-sys-33d79b7b3a6a5533/out/lib:/work/oxidecomputer/omicron/target/debug/build/ring-84313340968b144b/out:/work/oxidecomputer/omicron/target/debug/build/ring-ae424a22a6f8daa6/out:/work/oxidecomputer/omicron/target/debug/build/tofino-b9675a1b9b2f961b/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib NEXTEST_RUN_ID=b150fa8a-3f10-48fe-8ae3-3092cf265384 OUT_DIR=/work/oxidecomputer/omicron/target/debug/build/omicron-nexus-b4d57d26713e7bf8/out PATH=/work/oxidecomputer/omicron/out/mgd/root/opt/oxide/mgd/bin:/work/oxidecomputer/omicron/out/dendrite-stub/bin:/work/oxidecomputer/omicron/out/clickhouse:/work/oxidecomputer/omicron/out/cockroachdb/bin:/home/build/.cargo/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/ooce/bin:/opt/ooce/sbin PWD=/work/oxidecomputer/omicron RUSTC_BOOTSTRAP=1 RUSTDOCFLAGS=--document-private-items -D warnings RUSTFLAGS=-D warnings RUSTUP_HOME=/home/build/.rustup RUSTUP_TOOLCHAIN=1.81.0-x86_64-unknown-linux-gnu RUST_BACKTRACE=1 RUST_RECURSION_COUNT=1 SHLVL=1 SSL_CERT_DIR=/usr/lib/ssl/certs SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt TMPDIR=/var/tmp/omicron_tmp TZ=UTC USER=build __NEXTEST_ATTEMPT=1 __NEXTEST_ORIGINAL_CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus
72024-11-15T20:11:20.557ZINFOtest_mgs_metrics: cockroach pid: 155479
82024-11-15T20:11:20.557ZINFOtest_mgs_metrics: cockroach listen URL: postgresql://root@[::1]:46169/omicron?sslmode=disable
92024-11-15T20:11:20.557ZDEBGtest_mgs_metrics (init_with_steps): Running step start_clickhouse
102024-11-15T20:11:20.557ZDEBGtest_mgs_metrics: Starting Clickhouse
112024-11-15T20:11:21.479ZDEBGtest_mgs_metrics (init_with_steps): Running step start_gateway_switch0
122024-11-15T20:11:21.479ZDEBGtest_mgs_metrics: Starting Management Gateway
132024-11-15T20:11:21.695ZDEBGtest_mgs_metrics (init_with_steps): Running step start_gateway_switch1
142024-11-15T20:11:21.695ZDEBGtest_mgs_metrics: Starting Management Gateway
152024-11-15T20:11:21.899ZDEBGtest_mgs_metrics (init_with_steps): Running step start_dendrite_switch0
162024-11-15T20:11:21.899ZDEBGtest_mgs_metrics: Starting Dendrite for switch0
172024-11-15T20:11:21.958ZDEBGtest_mgs_metrics (init_with_steps): Running step start_dendrite_switch1
182024-11-15T20:11:21.958ZDEBGtest_mgs_metrics: Starting Dendrite for switch1
192024-11-15T20:11:22.035ZDEBGtest_mgs_metrics (init_with_steps): Running step start_mgd_switch0
202024-11-15T20:11:22.036ZDEBGtest_mgs_metrics: Starting mgd for switch0
212024-11-15T20:11:22.087ZDEBGtest_mgs_metrics: mgd port is 43149
222024-11-15T20:11:22.087ZDEBGtest_mgs_metrics (init_with_steps): Running step start_mgd_switch1
232024-11-15T20:11:22.087ZDEBGtest_mgs_metrics: Starting mgd for switch1
242024-11-15T20:11:22.144ZDEBGtest_mgs_metrics: mgd port is 41187
252024-11-15T20:11:22.144ZDEBGtest_mgs_metrics (init_with_steps): Running step record_switch_dns
262024-11-15T20:11:22.144ZDEBGtest_mgs_metrics: Recording DNS for the switch zones
272024-11-15T20:11:22.144ZDEBGtest_mgs_metrics (init_with_steps): Running step start_internal_dns
282024-11-15T20:11:22.144ZINFOtest_mgs_metrics (store): opening sled database path = /var/tmp/omicron_tmp/.tmpdaIVcI
292024-11-15T20:11:22.206ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
302024-11-15T20:11:22.206ZINFOtest_mgs_metrics (store): pruning trees for generations older than 0 keep = 3
312024-11-15T20:11:22.206ZINFOtest_mgs_metrics (dns): DNS server bound to address kind = dns local_address = [::1]:59900
322024-11-15T20:11:22.213ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:36837 method = GET path = /config versions = all
332024-11-15T20:11:22.213ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:36837 method = PUT path = /config versions = all
342024-11-15T20:11:22.214ZINFOtest_mgs_metrics (http): listening kind = dns local_addr = [::1]:36837
352024-11-15T20:11:22.214ZDEBGtest_mgs_metrics (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:36837
362024-11-15T20:11:22.214ZDEBGtest_mgs_metrics (init_with_steps): Running step start_external_dns
372024-11-15T20:11:22.214ZINFOtest_mgs_metrics (store): opening sled database path = /var/tmp/omicron_tmp/.tmpob0OrL
382024-11-15T20:11:22.214ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
392024-11-15T20:11:22.214ZINFOtest_mgs_metrics (store): pruning trees for generations older than 0 keep = 3
402024-11-15T20:11:22.215ZINFOtest_mgs_metrics (dns): DNS server bound to address kind = dns local_address = [::1]:37050
412024-11-15T20:11:22.215ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:40887 method = GET path = /config versions = all
422024-11-15T20:11:22.215ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:40887 method = PUT path = /config versions = all
432024-11-15T20:11:22.215ZINFOtest_mgs_metrics (http): listening kind = dns local_addr = [::1]:40887
442024-11-15T20:11:22.215ZDEBGtest_mgs_metrics (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:40887
452024-11-15T20:11:22.215ZDEBGtest_mgs_metrics (init_with_steps): Running step start_nexus_internal
462024-11-15T20:11:22.215ZDEBGtest_mgs_metrics: Starting Nexus (internal API)
472024-11-15T20:11:22.217ZINFOtest_mgs_metrics: setting up nexus server
482024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Action
492024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AnyActor
502024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AuthenticatedActor
512024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = BlueprintConfig
522024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Database
532024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DnsConfig
542024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Fleet
552024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Inventory
562024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IpPoolList
572024-11-15T20:11:22.217ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ConsoleSessionList
582024-11-15T20:11:22.218ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAuthRequestList
592024-11-15T20:11:22.218ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloCertificateList
602024-11-15T20:11:22.218ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloIdentityProviderList
612024-11-15T20:11:22.218ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloUserList
622024-11-15T20:11:22.218ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Project
632024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Disk
642024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Snapshot
652024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ProjectImage
662024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Instance
672024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IpPool
682024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InstanceNetworkInterface
692024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Vpc
702024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = VpcRouter
712024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGateway
722024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGatewayIpPool
732024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGatewayIpAddress
742024-11-15T20:11:22.219ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = RouterRoute
752024-11-15T20:11:22.220ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = VpcSubnet
762024-11-15T20:11:22.220ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = FloatingIp
772024-11-15T20:11:22.220ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Image
782024-11-15T20:11:22.220ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloImage
792024-11-15T20:11:22.220ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AddressLot
802024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Blueprint
812024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = LoopbackAddress
822024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Certificate
832024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ConsoleSession
842024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAuthRequest
852024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAccessToken
862024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = PhysicalDisk
872024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Rack
882024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = RoleBuiltin
892024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SshKey
902024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Silo
912024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloUser
922024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloGroup
932024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IdentityProvider
942024-11-15T20:11:22.221ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SamlIdentityProvider
952024-11-15T20:11:22.222ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Sled
962024-11-15T20:11:22.222ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = TufRepo
972024-11-15T20:11:22.222ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = TufArtifact
982024-11-15T20:11:22.222ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Zpool
992024-11-15T20:11:22.222ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Service
1002024-11-15T20:11:22.222ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = UserBuiltin
1012024-11-15T20:11:22.222ZINFOtest_mgs_metrics (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
1022024-11-15T20:11:22.241ZINFOtest_mgs_metrics (ServerContext): Setting up resolver using DNS address: [::1]:59900
1032024-11-15T20:11:22.241ZINFOtest_mgs_metrics (DnsResolver): new DNS resolver addresses = [[::1]:59900]
1042024-11-15T20:11:22.241ZINFOtest_mgs_metrics (ServerContext): Setting up qorb database pool from a single host url = PostgresConfigWithUrl {\n url_raw: "postgresql://root@[::1]:46169/omicron?sslmode=disable",\n config: Config {\n user: Some(\n "root",\n ),\n password: None,\n dbname: Some(\n "omicron",\n ),\n options: None,\n application_name: None,\n ssl_mode: Disable,\n host: [\n Tcp(\n "::1",\n ),\n ],\n hostaddr: [],\n port: [\n 46169,\n ],\n connect_timeout: None,\n tcp_user_timeout: None,\n keepalives: true,\n keepalives_idle: 7200s,\n keepalives_interval: None,\n keepalives_retries: None,\n target_session_attrs: Any,\n channel_binding: Prefer,\n },\n}
1052024-11-15T20:11:22.241ZDEBGtest_mgs_metrics (ServerContext): registered USDT probes
1062024-11-15T20:11:22.329ZINFOtest_mgs_metrics (nexus): Database schema version is up to date desired_version = 114.0.0 found_version = 114.0.0
1072024-11-15T20:11:22.655ZDEBGtest_mgs_metrics (clickhouse-client): registered USDT probes id = cf71258c-2316-4088-9145-f927ca712583
1082024-11-15T20:11:22.664ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /artifacts/{kind}/{name}/{version} versions = all
1092024-11-15T20:11:22.665ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /bgtasks versions = all
1102024-11-15T20:11:22.665ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /bgtasks/activate versions = all
1112024-11-15T20:11:22.665ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /bgtasks/view/{bgtask_name} versions = all
1122024-11-15T20:11:22.665ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /clickhouse/policy versions = all
1132024-11-15T20:11:22.665ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /clickhouse/policy versions = all
1142024-11-15T20:11:22.665ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
1152024-11-15T20:11:22.665ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
1162024-11-15T20:11:22.665ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
1172024-11-15T20:11:22.666ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1182024-11-15T20:11:22.666ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1192024-11-15T20:11:22.666ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /demo-saga versions = all
1202024-11-15T20:11:22.666ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1212024-11-15T20:11:22.666ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /deployment/blueprints/all versions = all
1222024-11-15T20:11:22.666ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1232024-11-15T20:11:22.666ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1242024-11-15T20:11:22.666ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /deployment/blueprints/import versions = all
1252024-11-15T20:11:22.667ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /deployment/blueprints/regenerate versions = all
1262024-11-15T20:11:22.667ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /deployment/blueprints/target versions = all
1272024-11-15T20:11:22.667ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /deployment/blueprints/target versions = all
1282024-11-15T20:11:22.667ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = PUT path = /deployment/blueprints/target/enabled versions = all
1292024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1302024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = PUT path = /disks/{disk_id} versions = all
1312024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /instances/{instance_id}/migrate versions = all
1322024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /metrics/collectors versions = all
1332024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1342024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /metrics/producers versions = all
1352024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1362024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /physical-disk/expunge versions = all
1372024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /probes/{sled} versions = all
1382024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1392024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /sagas versions = all
1402024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /sagas/{saga_id} versions = all
1412024-11-15T20:11:22.668ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /sled-agents/{sled_id} versions = all
1422024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /sled-agents/{sled_id} versions = all
1432024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1442024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /sleds/add versions = all
1452024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /sleds/expunge versions = all
1462024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = GET path = /sleds/uninitialized versions = all
1472024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = PUT path = /switch/{switch_id} versions = all
1482024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = PUT path = /vmms/{propolis_id} versions = all
1492024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:37069 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1502024-11-15T20:11:22.669ZINFOtest_mgs_metrics (dropshot_internal): listening local_addr = 127.0.0.1:37069
1512024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (dropshot_internal): successfully registered DTrace USDT probes local_addr = 127.0.0.1:37069
1522024-11-15T20:11:22.669ZINFOtest_mgs_metrics (SEC): SEC running sec_id = c74391b6-2721-445b-b829-94c88c82a5d0
1532024-11-15T20:11:22.669ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1542024-11-15T20:11:22.669ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1552024-11-15T20:11:22.672ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1562024-11-15T20:11:22.672ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in users
1572024-11-15T20:11:22.672ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1582024-11-15T20:11:22.673ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1592024-11-15T20:11:22.673ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1602024-11-15T20:11:22.688ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 82 f9 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:59632 req_id = a7c1df83-4404-49af-981d-6f3b1b217433
1612024-11-15T20:11:22.688ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33529,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59632 req_id = a7c1df83-4404-49af-981d-6f3b1b217433
1622024-11-15T20:11:22.688ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33529, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_crucible-pantry._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 16, 95, 99, 114, 117, 99, 105, 98, 108, 101, 45, 112, 97, 110, 116, 114, 121, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_crucible-pantry._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59632 req_id = a7c1df83-4404-49af-981d-6f3b1b217433
1632024-11-15T20:11:22.695ZINFOtest_mgs_metrics (DataLoader): created 5 built-in users
1642024-11-15T20:11:22.695ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1652024-11-15T20:11:22.695ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1662024-11-15T20:11:22.701ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1672024-11-15T20:11:22.701ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in roles
1682024-11-15T20:11:22.704ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1692024-11-15T20:11:22.707ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1702024-11-15T20:11:22.707ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1712024-11-15T20:11:22.739ZINFOtest_mgs_metrics (DataLoader): created 10 built-in roles
1722024-11-15T20:11:22.739ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1732024-11-15T20:11:22.739ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1742024-11-15T20:11:22.742ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1752024-11-15T20:11:22.742ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in role assignments
1762024-11-15T20:11:22.742ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1772024-11-15T20:11:22.743ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1782024-11-15T20:11:22.743ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1792024-11-15T20:11:23.363ZINFOtest_mgs_metrics (DataLoader): created 4 built-in role assignments
1802024-11-15T20:11:23.363ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1812024-11-15T20:11:23.363ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1822024-11-15T20:11:23.365ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1832024-11-15T20:11:23.366ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in silos
1842024-11-15T20:11:23.366ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1852024-11-15T20:11:23.366ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1862024-11-15T20:11:23.366ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1872024-11-15T20:11:23.446ZINFOtest_mgs_metrics (DataLoader): created 2 built-in silos
1882024-11-15T20:11:23.446ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1892024-11-15T20:11:23.446ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1902024-11-15T20:11:23.447ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1912024-11-15T20:11:23.473ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1922024-11-15T20:11:23.473ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1932024-11-15T20:11:23.474ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1942024-11-15T20:11:23.474ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in projects
1952024-11-15T20:11:23.474ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1962024-11-15T20:11:23.475ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1972024-11-15T20:11:23.475ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1982024-11-15T20:11:23.479ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
1992024-11-15T20:11:23.479ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2002024-11-15T20:11:23.479ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2012024-11-15T20:11:23.479ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2022024-11-15T20:11:23.480ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2032024-11-15T20:11:23.562ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2042024-11-15T20:11:23.562ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2052024-11-15T20:11:23.563ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2062024-11-15T20:11:23.563ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2072024-11-15T20:11:23.566ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2082024-11-15T20:11:23.570ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2092024-11-15T20:11:23.571ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2102024-11-15T20:11:23.571ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2112024-11-15T20:11:23.571ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2122024-11-15T20:11:23.571ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2132024-11-15T20:11:23.572ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2142024-11-15T20:11:23.575ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2152024-11-15T20:11:23.576ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2162024-11-15T20:11:23.576ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2172024-11-15T20:11:23.577ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2182024-11-15T20:11:23.579ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2192024-11-15T20:11:23.584ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2202024-11-15T20:11:23.584ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2212024-11-15T20:11:23.584ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2222024-11-15T20:11:23.587ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2232024-11-15T20:11:23.631ZINFOtest_mgs_metrics (DataLoader): created built-in services project
2242024-11-15T20:11:23.631ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2252024-11-15T20:11:23.632ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2262024-11-15T20:11:23.634ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2272024-11-15T20:11:23.634ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPCs
2282024-11-15T20:11:23.634ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2292024-11-15T20:11:23.634ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2302024-11-15T20:11:23.635ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2312024-11-15T20:11:23.664ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2322024-11-15T20:11:23.664ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2332024-11-15T20:11:23.665ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2342024-11-15T20:11:23.673ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2352024-11-15T20:11:23.677ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2362024-11-15T20:11:23.678ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2372024-11-15T20:11:23.678ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2382024-11-15T20:11:23.678ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2392024-11-15T20:11:23.683ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2402024-11-15T20:11:23.683ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2412024-11-15T20:11:23.683ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2422024-11-15T20:11:23.685ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2432024-11-15T20:11:23.687ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2442024-11-15T20:11:23.687ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2452024-11-15T20:11:23.688ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2462024-11-15T20:11:23.689ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2472024-11-15T20:11:23.691ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2482024-11-15T20:11:23.699ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2492024-11-15T20:11:23.699ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Project" using built-in user
2502024-11-15T20:11:23.699ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2512024-11-15T20:11:23.699ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2522024-11-15T20:11:23.699ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2532024-11-15T20:11:23.700ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2542024-11-15T20:11:23.701ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2552024-11-15T20:11:23.705ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2562024-11-15T20:11:23.706ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2572024-11-15T20:11:23.706ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2582024-11-15T20:11:23.707ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2592024-11-15T20:11:23.723ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2602024-11-15T20:11:23.724ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2612024-11-15T20:11:23.733ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2622024-11-15T20:11:23.734ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2632024-11-15T20:11:23.734ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2642024-11-15T20:11:23.747ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2652024-11-15T20:11:23.747ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2662024-11-15T20:11:23.747ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2672024-11-15T20:11:23.748ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2682024-11-15T20:11:23.803ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2692024-11-15T20:11:23.803ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2702024-11-15T20:11:23.803ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2712024-11-15T20:11:23.803ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2722024-11-15T20:11:23.803ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2732024-11-15T20:11:23.806ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2742024-11-15T20:11:23.807ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2752024-11-15T20:11:23.807ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2762024-11-15T20:11:23.807ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772024-11-15T20:11:23.810ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2782024-11-15T20:11:23.811ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2792024-11-15T20:11:23.811ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2802024-11-15T20:11:23.811ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2812024-11-15T20:11:23.844ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2822024-11-15T20:11:23.852ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2832024-11-15T20:11:23.853ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2842024-11-15T20:11:23.853ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2852024-11-15T20:11:23.854ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2862024-11-15T20:11:23.872ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2872024-11-15T20:11:23.872ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2882024-11-15T20:11:23.874ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2892024-11-15T20:11:23.900ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2902024-11-15T20:11:23.900ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2912024-11-15T20:11:23.901ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2922024-11-15T20:11:23.933ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2932024-11-15T20:11:23.934ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2942024-11-15T20:11:23.934ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2952024-11-15T20:11:23.934ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2962024-11-15T20:11:23.935ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2972024-11-15T20:11:23.937ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2982024-11-15T20:11:23.937ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2992024-11-15T20:11:23.938ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3002024-11-15T20:11:23.939ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3012024-11-15T20:11:23.941ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3022024-11-15T20:11:23.941ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3032024-11-15T20:11:23.941ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3042024-11-15T20:11:23.942ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3052024-11-15T20:11:23.951ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3062024-11-15T20:11:23.960ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3072024-11-15T20:11:23.960ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3082024-11-15T20:11:23.960ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3092024-11-15T20:11:23.961ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3102024-11-15T20:11:24.051ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3112024-11-15T20:11:24.051ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3122024-11-15T20:11:24.051ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3132024-11-15T20:11:24.051ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3142024-11-15T20:11:24.052ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152024-11-15T20:11:24.054ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3162024-11-15T20:11:24.054ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3172024-11-15T20:11:24.054ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3182024-11-15T20:11:24.054ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192024-11-15T20:11:24.064ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3202024-11-15T20:11:24.065ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3212024-11-15T20:11:24.065ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3222024-11-15T20:11:24.066ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3232024-11-15T20:11:24.068ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3242024-11-15T20:11:24.105ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3252024-11-15T20:11:24.105ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3262024-11-15T20:11:24.105ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3272024-11-15T20:11:24.105ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3282024-11-15T20:11:24.240ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3292024-11-15T20:11:24.240ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3302024-11-15T20:11:24.240ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3312024-11-15T20:11:24.240ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3322024-11-15T20:11:24.240ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332024-11-15T20:11:24.240ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3342024-11-15T20:11:24.240ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3352024-11-15T20:11:24.241ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3362024-11-15T20:11:24.241ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372024-11-15T20:11:24.241ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3382024-11-15T20:11:24.242ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3392024-11-15T20:11:24.242ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3402024-11-15T20:11:24.242ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3412024-11-15T20:11:24.243ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3422024-11-15T20:11:24.250ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3432024-11-15T20:11:24.252ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3442024-11-15T20:11:24.252ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3452024-11-15T20:11:24.252ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3462024-11-15T20:11:24.268ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPC firewall rules
3472024-11-15T20:11:24.268ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3482024-11-15T20:11:24.268ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3492024-11-15T20:11:24.269ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3502024-11-15T20:11:24.284ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3512024-11-15T20:11:24.284ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3522024-11-15T20:11:24.284ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3532024-11-15T20:11:24.290ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3542024-11-15T20:11:24.291ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3552024-11-15T20:11:24.291ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3562024-11-15T20:11:24.291ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3572024-11-15T20:11:24.291ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3582024-11-15T20:11:24.291ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3592024-11-15T20:11:24.291ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3602024-11-15T20:11:24.291ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3612024-11-15T20:11:24.291ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3622024-11-15T20:11:24.291ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3632024-11-15T20:11:24.291ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3642024-11-15T20:11:24.292ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3652024-11-15T20:11:24.293ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3662024-11-15T20:11:24.293ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3672024-11-15T20:11:24.293ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3682024-11-15T20:11:24.294ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3692024-11-15T20:11:24.295ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3702024-11-15T20:11:24.302ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3712024-11-15T20:11:24.302ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
3722024-11-15T20:11:24.303ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3732024-11-15T20:11:24.303ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3742024-11-15T20:11:24.303ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3752024-11-15T20:11:24.303ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3762024-11-15T20:11:24.304ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3772024-11-15T20:11:24.307ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3782024-11-15T20:11:24.308ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3792024-11-15T20:11:24.308ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3802024-11-15T20:11:24.308ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3812024-11-15T20:11:24.309ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3822024-11-15T20:11:24.310ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3832024-11-15T20:11:24.310ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3842024-11-15T20:11:24.310ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3852024-11-15T20:11:24.312ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3862024-11-15T20:11:24.322ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3872024-11-15T20:11:24.322ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3882024-11-15T20:11:24.322ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3892024-11-15T20:11:24.323ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3902024-11-15T20:11:24.336ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3912024-11-15T20:11:24.336ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3922024-11-15T20:11:24.336ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3932024-11-15T20:11:24.337ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3942024-11-15T20:11:24.338ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952024-11-15T20:11:24.341ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3962024-11-15T20:11:24.350ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3972024-11-15T20:11:24.350ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3982024-11-15T20:11:24.350ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992024-11-15T20:11:24.351ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4002024-11-15T20:11:24.352ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4012024-11-15T20:11:24.352ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4022024-11-15T20:11:24.352ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4032024-11-15T20:11:24.352ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4042024-11-15T20:11:24.357ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4052024-11-15T20:11:24.357ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4062024-11-15T20:11:24.357ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4072024-11-15T20:11:24.358ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4082024-11-15T20:11:24.490ZINFOtest_mgs_metrics (DataLoader): created built-in VPC firewall rules
4092024-11-15T20:11:24.490ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPC Subnets
4102024-11-15T20:11:24.490ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4112024-11-15T20:11:24.490ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4122024-11-15T20:11:24.490ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4132024-11-15T20:11:24.502ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4142024-11-15T20:11:24.502ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4152024-11-15T20:11:24.502ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4162024-11-15T20:11:24.505ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4172024-11-15T20:11:24.505ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4182024-11-15T20:11:24.505ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4192024-11-15T20:11:24.510ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4202024-11-15T20:11:24.510ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4212024-11-15T20:11:24.511ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4222024-11-15T20:11:24.511ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4232024-11-15T20:11:24.511ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4242024-11-15T20:11:24.517ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4252024-11-15T20:11:24.517ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4262024-11-15T20:11:24.517ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4272024-11-15T20:11:24.517ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4282024-11-15T20:11:24.522ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4292024-11-15T20:11:24.522ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4302024-11-15T20:11:24.523ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4312024-11-15T20:11:24.526ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4322024-11-15T20:11:24.526ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4332024-11-15T20:11:24.531ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4342024-11-15T20:11:24.534ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
4352024-11-15T20:11:24.534ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4362024-11-15T20:11:24.534ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4372024-11-15T20:11:24.535ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4382024-11-15T20:11:24.545ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4392024-11-15T20:11:24.545ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4402024-11-15T20:11:24.545ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4412024-11-15T20:11:24.545ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4422024-11-15T20:11:24.546ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4432024-11-15T20:11:24.549ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4442024-11-15T20:11:24.550ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4452024-11-15T20:11:24.550ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4462024-11-15T20:11:24.550ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4472024-11-15T20:11:24.552ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4482024-11-15T20:11:24.552ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4492024-11-15T20:11:24.552ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4502024-11-15T20:11:24.553ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4512024-11-15T20:11:24.555ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4522024-11-15T20:11:24.560ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4532024-11-15T20:11:24.580ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4542024-11-15T20:11:24.580ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4552024-11-15T20:11:24.582ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4562024-11-15T20:11:24.688ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4572024-11-15T20:11:24.688ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4582024-11-15T20:11:24.690ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4592024-11-15T20:11:24.715ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4602024-11-15T20:11:24.715ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4612024-11-15T20:11:24.715ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4622024-11-15T20:11:24.715ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4632024-11-15T20:11:24.716ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4642024-11-15T20:11:24.770ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4652024-11-15T20:11:24.770ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4662024-11-15T20:11:24.770ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4672024-11-15T20:11:24.770ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4682024-11-15T20:11:24.775ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4692024-11-15T20:11:24.776ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4702024-11-15T20:11:24.776ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4712024-11-15T20:11:24.776ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4722024-11-15T20:11:24.782ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4732024-11-15T20:11:24.792ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
4742024-11-15T20:11:24.796ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4752024-11-15T20:11:24.797ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4762024-11-15T20:11:24.797ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4772024-11-15T20:11:24.830ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4782024-11-15T20:11:24.830ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4792024-11-15T20:11:24.830ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4802024-11-15T20:11:24.834ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4812024-11-15T20:11:24.834ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4822024-11-15T20:11:24.834ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4832024-11-15T20:11:24.834ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4842024-11-15T20:11:24.835ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4852024-11-15T20:11:24.852ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4862024-11-15T20:11:24.852ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4872024-11-15T20:11:24.852ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4882024-11-15T20:11:24.880ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4892024-11-15T20:11:24.901ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4902024-11-15T20:11:24.901ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4912024-11-15T20:11:24.902ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4922024-11-15T20:11:24.902ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4932024-11-15T20:11:24.905ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4942024-11-15T20:11:24.912ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4952024-11-15T20:11:24.923ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4962024-11-15T20:11:24.924ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4972024-11-15T20:11:24.924ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4982024-11-15T20:11:24.953ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4992024-11-15T20:11:24.954ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5002024-11-15T20:11:24.955ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5012024-11-15T20:11:24.964ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5022024-11-15T20:11:24.969ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5032024-11-15T20:11:24.969ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5042024-11-15T20:11:24.969ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5052024-11-15T20:11:24.971ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5062024-11-15T20:11:24.979ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5072024-11-15T20:11:24.980ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5082024-11-15T20:11:24.980ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5092024-11-15T20:11:24.980ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5102024-11-15T20:11:24.982ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5112024-11-15T20:11:24.982ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5122024-11-15T20:11:24.982ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5132024-11-15T20:11:24.982ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5142024-11-15T20:11:24.985ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5152024-11-15T20:11:24.996ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5162024-11-15T20:11:24.996ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5172024-11-15T20:11:24.997ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5182024-11-15T20:11:24.997ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5192024-11-15T20:11:25.014ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5202024-11-15T20:11:25.014ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5212024-11-15T20:11:25.015ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5222024-11-15T20:11:25.021ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
5232024-11-15T20:11:25.021ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5242024-11-15T20:11:25.021ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5252024-11-15T20:11:25.021ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5262024-11-15T20:11:25.022ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5272024-11-15T20:11:25.026ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5282024-11-15T20:11:25.026ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5292024-11-15T20:11:25.026ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5302024-11-15T20:11:25.029ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5312024-11-15T20:11:25.035ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5322024-11-15T20:11:25.035ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5332024-11-15T20:11:25.035ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5342024-11-15T20:11:25.036ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5352024-11-15T20:11:25.039ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5362024-11-15T20:11:25.044ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5372024-11-15T20:11:25.052ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5382024-11-15T20:11:25.053ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5392024-11-15T20:11:25.056ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5402024-11-15T20:11:25.099ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5412024-11-15T20:11:25.100ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5422024-11-15T20:11:25.100ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5432024-11-15T20:11:25.116ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5442024-11-15T20:11:25.116ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5452024-11-15T20:11:25.116ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5462024-11-15T20:11:25.116ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5472024-11-15T20:11:25.117ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5482024-11-15T20:11:25.118ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5492024-11-15T20:11:25.118ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5502024-11-15T20:11:25.118ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5512024-11-15T20:11:25.119ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5522024-11-15T20:11:25.123ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5532024-11-15T20:11:25.123ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5542024-11-15T20:11:25.123ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5552024-11-15T20:11:25.123ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5562024-11-15T20:11:25.125ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5572024-11-15T20:11:25.135ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5582024-11-15T20:11:25.135ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5592024-11-15T20:11:25.135ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5602024-11-15T20:11:25.136ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5612024-11-15T20:11:25.146ZINFOtest_mgs_metrics (DataLoader): created built-in services vpc subnets
5622024-11-15T20:11:25.146ZINFOtest_mgs_metrics (DataLoader): created built-in services vpc
5632024-11-15T20:11:25.146ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5642024-11-15T20:11:25.146ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5652024-11-15T20:11:25.150ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5662024-11-15T20:11:25.150ZDEBGtest_mgs_metrics (DataLoader): attempting to create silo users
5672024-11-15T20:11:25.150ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5682024-11-15T20:11:25.150ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5692024-11-15T20:11:25.152ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5702024-11-15T20:11:25.167ZINFOtest_mgs_metrics (DataLoader): created 2 silo users
5712024-11-15T20:11:25.168ZDEBGtest_mgs_metrics (DataLoader): attempting to create silo user role assignments
5722024-11-15T20:11:25.168ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5732024-11-15T20:11:25.168ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5742024-11-15T20:11:25.169ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5752024-11-15T20:11:25.177ZINFOtest_mgs_metrics (DataLoader): created 2 silo user role assignments
5762024-11-15T20:11:25.178ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5772024-11-15T20:11:25.178ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5782024-11-15T20:11:25.178ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5792024-11-15T20:11:25.187ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5802024-11-15T20:11:25.188ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5812024-11-15T20:11:25.188ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5822024-11-15T20:11:25.206ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5832024-11-15T20:11:25.207ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5842024-11-15T20:11:25.207ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5852024-11-15T20:11:25.207ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5862024-11-15T20:11:25.207ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5872024-11-15T20:11:25.218ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5882024-11-15T20:11:25.223ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5892024-11-15T20:11:25.223ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5902024-11-15T20:11:25.223ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5912024-11-15T20:11:25.224ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5922024-11-15T20:11:25.248ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5932024-11-15T20:11:25.248ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5942024-11-15T20:11:25.248ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5952024-11-15T20:11:25.248ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5962024-11-15T20:11:25.248ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5972024-11-15T20:11:25.248ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5982024-11-15T20:11:25.250ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5992024-11-15T20:11:25.250ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6002024-11-15T20:11:25.250ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
6012024-11-15T20:11:25.250ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6022024-11-15T20:11:25.386ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6032024-11-15T20:11:25.387ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
6042024-11-15T20:11:25.387ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6052024-11-15T20:11:25.392ZINFOtest_mgs_metrics (nexus): populate complete
6062024-11-15T20:11:25.392ZINFOtest_mgs_metrics (nexus): activating background tasks
6072024-11-15T20:11:25.394ZDEBGtest_mgs_metrics (init_with_steps): Running step start_sled1
6082024-11-15T20:11:25.394ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6092024-11-15T20:11:25.449ZINFOtest_mgs_metrics (SledAgent): created simulated sled agent server = b6d65341-167c-41df-9b5c-41cded99c229 sim_mode = Explicit sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6102024-11-15T20:11:25.513ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /artifacts sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6112024-11-15T20:11:25.514ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = DELETE path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6122024-11-15T20:11:25.514ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6132024-11-15T20:11:25.514ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6142024-11-15T20:11:25.514ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6152024-11-15T20:11:25.514ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6162024-11-15T20:11:25.514ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6172024-11-15T20:11:25.514ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /bootstore/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6182024-11-15T20:11:25.515ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /cockroachdb sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6192024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6202024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6212024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /disks/{disk_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6222024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /disks/{disk_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6232024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /eip-gateways sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6242024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /inventory sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6252024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6262024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6272024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6282024-11-15T20:11:25.516ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6292024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /omicron-zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6302024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /sled-identifiers sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6312024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /sled-role sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6322024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /sleds sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6332024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /support/ipadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6342024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /support/zoneadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6352024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /switch-ports sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6362024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /timesync sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6372024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /update sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6382024-11-15T20:11:25.517ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = DELETE path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6392024-11-15T20:11:25.518ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6402024-11-15T20:11:25.518ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6412024-11-15T20:11:25.518ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = DELETE path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6422024-11-15T20:11:25.519ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6432024-11-15T20:11:25.519ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6442024-11-15T20:11:25.519ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6452024-11-15T20:11:25.519ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6462024-11-15T20:11:25.519ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /vmms/{propolis_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6472024-11-15T20:11:25.519ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6482024-11-15T20:11:25.519ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6492024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6502024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6512024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6522024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6532024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6542024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6552024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /zones/bundle-cleanup sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6562024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6572024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = PUT path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6582024-11-15T20:11:25.521ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /zones/bundle-cleanup/utilization sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6592024-11-15T20:11:25.522ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /zones/bundles sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6602024-11-15T20:11:25.522ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6612024-11-15T20:11:25.523ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = POST path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6622024-11-15T20:11:25.523ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6632024-11-15T20:11:25.523ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6642024-11-15T20:11:25.523ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:45047 method = GET path = /zpools sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6652024-11-15T20:11:25.523ZINFOtest_mgs_metrics (dropshot): listening local_addr = [::1]:45047 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6662024-11-15T20:11:25.523ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:45047 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6672024-11-15T20:11:25.523ZDEBGtest_mgs_metrics (omicron_sled_agent::sim::Server): contacting server nexus sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6682024-11-15T20:11:25.523ZDEBGtest_mgs_metrics (NexusClient): client request body = Some(Body) method = POST sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = http://127.0.0.1:37069/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
6692024-11-15T20:11:25.523ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44678
6702024-11-15T20:11:25.523ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
6712024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6722024-11-15T20:11:25.524ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6732024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6742024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6752024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
6762024-11-15T20:11:25.524ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
6772024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
6782024-11-15T20:11:25.524ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config nor servers
6792024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 44.513\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6802024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
6812024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6822024-11-15T20:11:25.524ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
6832024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6842024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6852024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
6862024-11-15T20:11:25.524ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
6872024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
6882024-11-15T20:11:25.524ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config nor servers
6892024-11-15T20:11:25.524ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 23.762\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6902024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
6912024-11-15T20:11:25.525ZINFOtest_mgs_metrics (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-15 20:01:25.518820051 UTC
6922024-11-15T20:11:25.525ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
6932024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6942024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6952024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
6962024-11-15T20:11:25.525ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
6972024-11-15T20:11:25.525ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6982024-11-15T20:11:25.525ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
6992024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7002024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7012024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
7022024-11-15T20:11:25.525ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
7032024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7042024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7052024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
7062024-11-15T20:11:25.525ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
7072024-11-15T20:11:25.525ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7082024-11-15T20:11:25.526ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7092024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
7102024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
7112024-11-15T20:11:25.527ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
7122024-11-15T20:11:25.527ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7132024-11-15T20:11:25.527ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
7142024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7152024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7162024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
7172024-11-15T20:11:25.527ZWARNtest_mgs_metrics (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = no blueprint
7182024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 36.482\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7192024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
7202024-11-15T20:11:25.527ZWARNtest_mgs_metrics (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector reason = no blueprint
7212024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 13.471\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7222024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
7232024-11-15T20:11:25.527ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
7242024-11-15T20:11:25.527ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7252024-11-15T20:11:25.527ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
7262024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7272024-11-15T20:11:25.527ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7282024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
7292024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 33.352\u{b5}s iteration = 1 status = {"error":"task disabled"}
7302024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
7312024-11-15T20:11:25.528ZINFOtest_mgs_metrics (BackgroundTasks): Decommissioned Disk Cleaner disabled background_task = decommissioned_disk_cleaner
7322024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 41.982\u{b5}s iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7332024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
7342024-11-15T20:11:25.528ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
7352024-11-15T20:11:25.528ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7362024-11-15T20:11:25.528ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
7372024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7382024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7392024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
7402024-11-15T20:11:25.528ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
7412024-11-15T20:11:25.528ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7422024-11-15T20:11:25.528ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
7432024-11-15T20:11:25.528ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7442024-11-15T20:11:25.529ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7452024-11-15T20:11:25.529ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
7462024-11-15T20:11:25.529ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
7472024-11-15T20:11:25.529ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7482024-11-15T20:11:25.530ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7492024-11-15T20:11:25.530ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
7502024-11-15T20:11:25.530ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
7512024-11-15T20:11:25.530ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7522024-11-15T20:11:25.531ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7532024-11-15T20:11:25.531ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
7542024-11-15T20:11:25.531ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
7552024-11-15T20:11:25.534ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7562024-11-15T20:11:25.535ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7572024-11-15T20:11:25.535ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
7582024-11-15T20:11:25.537ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
7592024-11-15T20:11:25.537ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7602024-11-15T20:11:25.537ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
7612024-11-15T20:11:25.537ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7622024-11-15T20:11:25.537ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7632024-11-15T20:11:25.537ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
7642024-11-15T20:11:25.537ZINFOtest_mgs_metrics (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
7652024-11-15T20:11:25.537ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_updater elapsed = 99.216\u{b5}s iteration = 1 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7662024-11-15T20:11:25.537ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
7672024-11-15T20:11:25.537ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
7682024-11-15T20:11:25.537ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7692024-11-15T20:11:25.537ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7702024-11-15T20:11:25.537ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
7712024-11-15T20:11:25.538ZDEBGtest_mgs_metrics (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7722024-11-15T20:11:25.538ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
7732024-11-15T20:11:25.538ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7742024-11-15T20:11:25.538ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7752024-11-15T20:11:25.538ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
7762024-11-15T20:11:25.538ZINFOtest_mgs_metrics (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
7772024-11-15T20:11:25.538ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
7782024-11-15T20:11:25.538ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7792024-11-15T20:11:25.539ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
7802024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7812024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7822024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
7832024-11-15T20:11:25.539ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
7842024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7852024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7862024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
7872024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (SagaRecoverer): listing candidate sagas for recovery
7882024-11-15T20:11:25.539ZTRACtest_mgs_metrics (SagaRecoverer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database
7892024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (SagaRecoverer): roles roles = RoleSet { roles: {} }
7902024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
7912024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
7922024-11-15T20:11:25.539ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
7932024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
7942024-11-15T20:11:25.539ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
7952024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
7962024-11-15T20:11:25.540ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
7972024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7982024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7992024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
8002024-11-15T20:11:25.540ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database
8012024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
8022024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8032024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
8042024-11-15T20:11:25.540ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
8052024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8062024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8072024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
8082024-11-15T20:11:25.540ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database
8092024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
8102024-11-15T20:11:25.540ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8112024-11-15T20:11:25.542ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44678 req_id = 1013c3bd-83f0-4396-b11d-07aff18d6302 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8122024-11-15T20:11:25.542ZINFOtest_mgs_metrics (nexus): registered sled agent sled_uuid = b6d65341-167c-41df-9b5c-41cded99c229
8132024-11-15T20:11:25.542ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
8142024-11-15T20:11:25.542ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8152024-11-15T20:11:25.542ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
8162024-11-15T20:11:25.542ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8172024-11-15T20:11:25.542ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8182024-11-15T20:11:25.542ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: ee c9 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 ............._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:65437 req_id = 4531198a-bb2b-421c-b842-f10fbc5bf7cd
8192024-11-15T20:11:25.542ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61129,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65437 req_id = 4531198a-bb2b-421c-b842-f10fbc5bf7cd
8202024-11-15T20:11:25.542ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61129, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_nameservice._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 12, 95, 110, 97, 109, 101, 115, 101, 114, 118, 105, 99, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_nameservice._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65437 req_id = 4531198a-bb2b-421c-b842-f10fbc5bf7cd
8212024-11-15T20:11:25.542ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 6e 8b 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 n............_ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55141 req_id = e895f3db-eaec-4935-922c-100f835aa92f
8222024-11-15T20:11:25.542ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28299,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55141 req_id = e895f3db-eaec-4935-922c-100f835aa92f
8232024-11-15T20:11:25.543ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28299, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_external-dns._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 13, 95, 101, 120, 116, 101, 114, 110, 97, 108, 45, 100, 110, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_external-dns._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55141 req_id = e895f3db-eaec-4935-922c-100f835aa92f
8242024-11-15T20:11:25.544ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
8252024-11-15T20:11:25.544ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8262024-11-15T20:11:25.544ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
8272024-11-15T20:11:25.544ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8282024-11-15T20:11:25.544ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8292024-11-15T20:11:25.544ZWARNtest_mgs_metrics (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_internal dns_group = internal error = no record found for Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8302024-11-15T20:11:25.544ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 26.511929ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_nameservice._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8312024-11-15T20:11:25.544ZWARNtest_mgs_metrics (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_external dns_group = external error = no record found for Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8322024-11-15T20:11:25.544ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 25.412299ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_external-dns._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8332024-11-15T20:11:25.559ZINFOtest_mgs_metrics (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-15 20:01:25.518820051 UTC pruned = {}
8342024-11-15T20:11:25.559ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 40.886154ms iteration = 1 status = {"expiration":"2024-11-15T20:01:25.518820051Z","pruned":[]}
8352024-11-15T20:11:25.559ZTRACtest_mgs_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8362024-11-15T20:11:25.561ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: c4 8c 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:49496 req_id = 4397840d-809b-45b1-a4de-ea58ceac3159
8372024-11-15T20:11:25.561ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50316,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49496 req_id = 4397840d-809b-45b1-a4de-ea58ceac3159
8382024-11-15T20:11:25.561ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50316, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49496 req_id = 4397840d-809b-45b1-a4de-ea58ceac3159
8392024-11-15T20:11:25.563ZERROtest_mgs_metrics (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8402024-11-15T20:11:25.564ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 43.184769ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8412024-11-15T20:11:25.574ZTRACtest_mgs_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8422024-11-15T20:11:25.575ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: cc 42 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .B..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54792 req_id = 83bf0652-5d45-4f89-8099-2cada10966fe
8432024-11-15T20:11:25.576ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52290,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54792 req_id = 83bf0652-5d45-4f89-8099-2cada10966fe
8442024-11-15T20:11:25.576ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52290, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54792 req_id = 83bf0652-5d45-4f89-8099-2cada10966fe
8452024-11-15T20:11:25.576ZERROtest_mgs_metrics (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8462024-11-15T20:11:25.576ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 54.869138ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8472024-11-15T20:11:25.601ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8482024-11-15T20:11:25.612ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8492024-11-15T20:11:25.612ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
8502024-11-15T20:11:25.613ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8512024-11-15T20:11:25.613ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8522024-11-15T20:11:25.614ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 92.690125ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8532024-11-15T20:11:25.660ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8542024-11-15T20:11:25.701ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8552024-11-15T20:11:25.704ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
8562024-11-15T20:11:25.704ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8572024-11-15T20:11:25.705ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8582024-11-15T20:11:25.705ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8592024-11-15T20:11:25.712ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8602024-11-15T20:11:25.712ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
8612024-11-15T20:11:25.712ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8622024-11-15T20:11:25.712ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8632024-11-15T20:11:25.771ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
8642024-11-15T20:11:25.771ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8652024-11-15T20:11:25.774ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8662024-11-15T20:11:25.784ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8672024-11-15T20:11:25.797ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8682024-11-15T20:11:25.797ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
8692024-11-15T20:11:25.797ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8702024-11-15T20:11:25.799ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8712024-11-15T20:11:25.800ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8722024-11-15T20:11:25.814ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8732024-11-15T20:11:25.814ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
8742024-11-15T20:11:25.814ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8752024-11-15T20:11:25.816ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8762024-11-15T20:11:25.864ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8772024-11-15T20:11:25.875ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8782024-11-15T20:11:25.876ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
8792024-11-15T20:11:25.877ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8802024-11-15T20:11:25.877ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
8812024-11-15T20:11:25.877ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8822024-11-15T20:11:25.877ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8832024-11-15T20:11:25.879ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8842024-11-15T20:11:25.888ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8852024-11-15T20:11:25.891ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
8862024-11-15T20:11:25.891ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8872024-11-15T20:11:25.891ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8882024-11-15T20:11:25.891ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
8892024-11-15T20:11:25.891ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8902024-11-15T20:11:25.891ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8912024-11-15T20:11:25.900ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 364.678916ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
8922024-11-15T20:11:25.935ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
8932024-11-15T20:11:25.935ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8942024-11-15T20:11:25.937ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8952024-11-15T20:11:25.982ZDEBGtest_mgs_metrics (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
8962024-11-15T20:11:25.982ZINFOtest_mgs_metrics (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
8972024-11-15T20:11:25.982ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 448.047012ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
8982024-11-15T20:11:25.987ZTRACtest_mgs_metrics (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = instance failed total_found = 0
8992024-11-15T20:11:25.987ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
9002024-11-15T20:11:25.987ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9012024-11-15T20:11:25.987ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9022024-11-15T20:11:26.045ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9032024-11-15T20:11:26.052ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9042024-11-15T20:11:26.053ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9052024-11-15T20:11:26.066ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9062024-11-15T20:11:26.072ZINFOtest_mgs_metrics (SagaRecoverer): listed in-progress sagas count = 0
9072024-11-15T20:11:26.072ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 536.945121ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
9082024-11-15T20:11:26.072ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker
9092024-11-15T20:11:26.072ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 548.593656ms iteration = 1 status = {"error":"inventory collection is None"}
9102024-11-15T20:11:26.072ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 534.923459ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
9112024-11-15T20:11:26.076ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
9122024-11-15T20:11:26.076ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9132024-11-15T20:11:26.077ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9142024-11-15T20:11:26.077ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9152024-11-15T20:11:26.077ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9162024-11-15T20:11:26.138ZWARNtest_mgs_metrics (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set
9172024-11-15T20:11:26.138ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 616.174279ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
9182024-11-15T20:11:26.142ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9192024-11-15T20:11:26.142ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9202024-11-15T20:11:26.145ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9212024-11-15T20:11:26.181ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 655.026012ms iteration = 1 status = {}
9222024-11-15T20:11:26.188ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
9232024-11-15T20:11:26.189ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9242024-11-15T20:11:26.189ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9252024-11-15T20:11:26.191ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9262024-11-15T20:11:26.196ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9272024-11-15T20:11:26.196ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9282024-11-15T20:11:26.196ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9292024-11-15T20:11:26.197ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9302024-11-15T20:11:26.205ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
9312024-11-15T20:11:26.206ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
9322024-11-15T20:11:26.206ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 30 92 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 0............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:50700 req_id = ba4aed36-c370-4ce2-8385-119b301fa29e
9332024-11-15T20:11:26.206ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12434,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50700 req_id = ba4aed36-c370-4ce2-8385-119b301fa29e
9342024-11-15T20:11:26.207ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 12434, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50700 req_id = ba4aed36-c370-4ce2-8385-119b301fa29e
9352024-11-15T20:11:26.207ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
9362024-11-15T20:11:26.207ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 684.343254ms iteration = 1 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
9372024-11-15T20:11:26.229ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9382024-11-15T20:11:26.235ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9392024-11-15T20:11:26.235ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9402024-11-15T20:11:26.236ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9412024-11-15T20:11:26.236ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9422024-11-15T20:11:26.284ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
9432024-11-15T20:11:26.284ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
9442024-11-15T20:11:26.285ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9452024-11-15T20:11:26.285ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9462024-11-15T20:11:26.285ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9472024-11-15T20:11:26.312ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 773.415412ms iteration = 1 status = {"errors":[],"records_set_to_done":[]}
9482024-11-15T20:11:26.337ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
9492024-11-15T20:11:26.337ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9502024-11-15T20:11:26.337ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9512024-11-15T20:11:26.373ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
9522024-11-15T20:11:26.373ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9532024-11-15T20:11:26.373ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9542024-11-15T20:11:26.373ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9552024-11-15T20:11:26.395ZWARNtest_mgs_metrics (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0
9562024-11-15T20:11:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 877.851648ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
9572024-11-15T20:11:26.397ZWARNtest_mgs_metrics (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0
9582024-11-15T20:11:26.397ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 880.715124ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
9592024-11-15T20:11:26.437ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
9602024-11-15T20:11:26.437ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
9612024-11-15T20:11:26.437ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9622024-11-15T20:11:26.437ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
9632024-11-15T20:11:26.437ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9642024-11-15T20:11:26.438ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9652024-11-15T20:11:26.439ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 898244 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44678 req_id = 1013c3bd-83f0-4396-b11d-07aff18d6302 response_code = 204 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
9662024-11-15T20:11:26.445ZDEBGtest_mgs_metrics (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:37069/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229", status: 204, headers: {"x-request-id": "1013c3bd-83f0-4396-b11d-07aff18d6302", "date": "Fri, 15 Nov 2024 20:11:25 GMT"} }) sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9672024-11-15T20:11:26.446ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9682024-11-15T20:11:26.446ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9692024-11-15T20:11:26.446ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9702024-11-15T20:11:26.446ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9712024-11-15T20:11:26.447ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9722024-11-15T20:11:26.447ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9732024-11-15T20:11:26.447ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9742024-11-15T20:11:26.447ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9752024-11-15T20:11:26.447ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35933 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9762024-11-15T20:11:26.447ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:35933 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9772024-11-15T20:11:26.448ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:35933 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9782024-11-15T20:11:26.448ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:35933 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9792024-11-15T20:11:26.448ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9802024-11-15T20:11:26.448ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9812024-11-15T20:11:26.449ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9822024-11-15T20:11:26.449ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9832024-11-15T20:11:26.449ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9842024-11-15T20:11:26.449ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9852024-11-15T20:11:26.449ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9862024-11-15T20:11:26.449ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9872024-11-15T20:11:26.449ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45427 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9882024-11-15T20:11:26.449ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45427 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9892024-11-15T20:11:26.450ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45427 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9902024-11-15T20:11:26.450ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45427 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9912024-11-15T20:11:26.450ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9922024-11-15T20:11:26.450ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9932024-11-15T20:11:26.450ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9942024-11-15T20:11:26.450ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9952024-11-15T20:11:26.450ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9962024-11-15T20:11:26.451ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9972024-11-15T20:11:26.451ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9982024-11-15T20:11:26.451ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9992024-11-15T20:11:26.451ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38911 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10002024-11-15T20:11:26.451ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38911 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10012024-11-15T20:11:26.452ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38911 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10022024-11-15T20:11:26.452ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38911 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10032024-11-15T20:11:26.452ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10042024-11-15T20:11:26.452ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10052024-11-15T20:11:26.453ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10062024-11-15T20:11:26.453ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10072024-11-15T20:11:26.453ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10082024-11-15T20:11:26.453ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10092024-11-15T20:11:26.453ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10102024-11-15T20:11:26.453ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10112024-11-15T20:11:26.453ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45423 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10122024-11-15T20:11:26.454ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45423 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10132024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45423 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10142024-11-15T20:11:26.454ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45423 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10152024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10162024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10172024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10182024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10192024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10202024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10212024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10222024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10232024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42169 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10242024-11-15T20:11:26.454ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:42169 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10252024-11-15T20:11:26.454ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:42169 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10262024-11-15T20:11:26.455ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:42169 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10272024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10282024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10292024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10302024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10312024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10322024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10332024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10342024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10352024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35587 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10362024-11-15T20:11:26.455ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:35587 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10372024-11-15T20:11:26.455ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:35587 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10382024-11-15T20:11:26.455ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:35587 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10392024-11-15T20:11:26.464ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10402024-11-15T20:11:26.464ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10412024-11-15T20:11:26.464ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10422024-11-15T20:11:26.464ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10432024-11-15T20:11:26.465ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10442024-11-15T20:11:26.465ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10452024-11-15T20:11:26.465ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10462024-11-15T20:11:26.465ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10472024-11-15T20:11:26.465ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40657 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10482024-11-15T20:11:26.465ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:40657 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10492024-11-15T20:11:26.465ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:40657 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10502024-11-15T20:11:26.466ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:40657 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10512024-11-15T20:11:26.466ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10522024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10532024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10542024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10552024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10562024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10572024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10582024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10592024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43137 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10602024-11-15T20:11:26.467ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43137 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10612024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43137 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10622024-11-15T20:11:26.467ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43137 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10632024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10642024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10652024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10662024-11-15T20:11:26.467ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10672024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10682024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10692024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10702024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10712024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39721 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10722024-11-15T20:11:26.468ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:39721 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10732024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:39721 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10742024-11-15T20:11:26.468ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:39721 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10752024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10762024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10772024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10782024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10792024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10802024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10812024-11-15T20:11:26.468ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10822024-11-15T20:11:26.469ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10832024-11-15T20:11:26.469ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38251 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10842024-11-15T20:11:26.469ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38251 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10852024-11-15T20:11:26.469ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38251 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10862024-11-15T20:11:26.469ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38251 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10872024-11-15T20:11:26.469ZDEBGtest_mgs_metrics (init_with_steps): Running step start_sled2
10882024-11-15T20:11:26.469ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10892024-11-15T20:11:26.533ZINFOtest_mgs_metrics (SledAgent): created simulated sled agent server = 039be560-54cc-49e3-88df-1a29dadbf913 sim_mode = Explicit sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
10902024-11-15T20:11:26.582ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /artifacts sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10912024-11-15T20:11:26.582ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = DELETE path = /artifacts/{sha256} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10922024-11-15T20:11:26.582ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /artifacts/{sha256} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10932024-11-15T20:11:26.582ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10942024-11-15T20:11:26.583ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10952024-11-15T20:11:26.583ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10962024-11-15T20:11:26.583ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10972024-11-15T20:11:26.583ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /bootstore/status sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10982024-11-15T20:11:26.583ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /cockroachdb sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
10992024-11-15T20:11:26.583ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /datasets sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11002024-11-15T20:11:26.583ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /datasets sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11012024-11-15T20:11:26.583ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /disks/{disk_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11022024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /disks/{disk_id}/poke sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11032024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /eip-gateways sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11042024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /inventory sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11052024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /network-bootstore-config sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11062024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /network-bootstore-config sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11072024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /omicron-physical-disks sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11082024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /omicron-physical-disks sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11092024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /omicron-zones sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11102024-11-15T20:11:26.584ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /sled-identifiers sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11112024-11-15T20:11:26.585ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /sled-role sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11122024-11-15T20:11:26.585ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /sleds sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11132024-11-15T20:11:26.585ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /support/ipadm-info sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11142024-11-15T20:11:26.585ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /support/zoneadm-info sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11152024-11-15T20:11:26.585ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /switch-ports sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11162024-11-15T20:11:26.585ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /timesync sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11172024-11-15T20:11:26.585ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /update sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11182024-11-15T20:11:26.585ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = DELETE path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11192024-11-15T20:11:26.586ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11202024-11-15T20:11:26.586ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11212024-11-15T20:11:26.586ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = DELETE path = /vmms/{propolis_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11222024-11-15T20:11:26.586ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /vmms/{propolis_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11232024-11-15T20:11:26.586ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11242024-11-15T20:11:26.587ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11252024-11-15T20:11:26.588ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11262024-11-15T20:11:26.588ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /vmms/{propolis_id}/poke sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11272024-11-15T20:11:26.588ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11282024-11-15T20:11:26.588ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11292024-11-15T20:11:26.588ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /vmms/{propolis_id}/state sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11302024-11-15T20:11:26.588ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /vmms/{propolis_id}/state sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11312024-11-15T20:11:26.588ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11322024-11-15T20:11:26.589ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /vpc-routes sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11332024-11-15T20:11:26.589ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /vpc-routes sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11342024-11-15T20:11:26.589ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /zones sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11352024-11-15T20:11:26.589ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /zones/bundle-cleanup sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11362024-11-15T20:11:26.589ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /zones/bundle-cleanup/context sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11372024-11-15T20:11:26.589ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = PUT path = /zones/bundle-cleanup/context sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11382024-11-15T20:11:26.590ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /zones/bundle-cleanup/utilization sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11392024-11-15T20:11:26.590ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /zones/bundles sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11402024-11-15T20:11:26.590ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /zones/bundles/{zone_name} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11412024-11-15T20:11:26.591ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = POST path = /zones/bundles/{zone_name} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11422024-11-15T20:11:26.591ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11432024-11-15T20:11:26.591ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11442024-11-15T20:11:26.591ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39103 method = GET path = /zpools sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11452024-11-15T20:11:26.591ZINFOtest_mgs_metrics (dropshot): listening local_addr = [::1]:39103 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11462024-11-15T20:11:26.592ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:39103 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11472024-11-15T20:11:26.592ZDEBGtest_mgs_metrics (omicron_sled_agent::sim::Server): contacting server nexus sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11482024-11-15T20:11:26.592ZDEBGtest_mgs_metrics (NexusClient): client request body = Some(Body) method = POST sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = http://127.0.0.1:37069/sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
11492024-11-15T20:11:26.593ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11502024-11-15T20:11:26.593ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11512024-11-15T20:11:26.594ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11522024-11-15T20:11:26.595ZINFOtest_mgs_metrics (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
11532024-11-15T20:11:26.595ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 1.062934362s iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
11542024-11-15T20:11:26.597ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_replacement elapsed = 1.06712652s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11552024-11-15T20:11:26.597ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 1.066065853s iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11562024-11-15T20:11:26.597ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11572024-11-15T20:11:26.597ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11582024-11-15T20:11:26.598ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11592024-11-15T20:11:26.598ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11602024-11-15T20:11:26.600ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 1.064338558s iteration = 1 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
11612024-11-15T20:11:26.601ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11622024-11-15T20:11:26.607ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11632024-11-15T20:11:26.607ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
11642024-11-15T20:11:26.608ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11652024-11-15T20:11:26.608ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11662024-11-15T20:11:26.609ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44680
11672024-11-15T20:11:26.610ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44680 req_id = c53bfbd3-f57d-4b50-bfad-9aa7806ab1ec uri = /sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
11682024-11-15T20:11:26.611ZINFOtest_mgs_metrics (nexus): registered sled agent sled_uuid = 039be560-54cc-49e3-88df-1a29dadbf913
11692024-11-15T20:11:26.612ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11702024-11-15T20:11:26.620ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11712024-11-15T20:11:26.620ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
11722024-11-15T20:11:26.620ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
11732024-11-15T20:11:26.620ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
11742024-11-15T20:11:26.620ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11752024-11-15T20:11:26.620ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11762024-11-15T20:11:26.623ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11772024-11-15T20:11:26.644ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
11782024-11-15T20:11:26.645ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
11792024-11-15T20:11:26.647ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 27 85 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 '............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52780 req_id = 40c09877-1f32-4062-ba62-cebcbb2380de
11802024-11-15T20:11:26.648ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10117,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52780 req_id = 40c09877-1f32-4062-ba62-cebcbb2380de
11812024-11-15T20:11:26.649ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10117, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52780 req_id = 40c09877-1f32-4062-ba62-cebcbb2380de
11822024-11-15T20:11:26.650ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
11832024-11-15T20:11:26.650ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 213.19424ms iteration = 2 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
11842024-11-15T20:11:26.691ZWARNtest_mgs_metrics (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone
11852024-11-15T20:11:26.691ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 1.171844082s iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
11862024-11-15T20:11:26.770ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Signaled
11872024-11-15T20:11:26.780ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
11882024-11-15T20:11:26.780ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11892024-11-15T20:11:26.780ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
11902024-11-15T20:11:26.780ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11912024-11-15T20:11:26.781ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11922024-11-15T20:11:26.783ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 172200 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44680 req_id = c53bfbd3-f57d-4b50-bfad-9aa7806ab1ec response_code = 204 uri = /sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
11932024-11-15T20:11:26.789ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11942024-11-15T20:11:26.800ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11952024-11-15T20:11:26.801ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
11962024-11-15T20:11:26.802ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11972024-11-15T20:11:26.802ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11982024-11-15T20:11:26.803ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
11992024-11-15T20:11:26.806ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12002024-11-15T20:11:26.806ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12012024-11-15T20:11:26.806ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12022024-11-15T20:11:26.806ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12032024-11-15T20:11:26.806ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12042024-11-15T20:11:26.807ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12052024-11-15T20:11:26.808ZDEBGtest_mgs_metrics (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:37069/sled-agents/039be560-54cc-49e3-88df-1a29dadbf913", status: 204, headers: {"x-request-id": "c53bfbd3-f57d-4b50-bfad-9aa7806ab1ec", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} }) sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12062024-11-15T20:11:26.812ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12072024-11-15T20:11:26.812ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12082024-11-15T20:11:26.812ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12092024-11-15T20:11:26.812ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12102024-11-15T20:11:26.812ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12112024-11-15T20:11:26.812ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12122024-11-15T20:11:26.813ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12132024-11-15T20:11:26.813ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12142024-11-15T20:11:26.813ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38511 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12152024-11-15T20:11:26.813ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38511 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12162024-11-15T20:11:26.813ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38511 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12172024-11-15T20:11:26.813ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38511 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12182024-11-15T20:11:26.815ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12192024-11-15T20:11:26.816ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12202024-11-15T20:11:26.816ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12212024-11-15T20:11:26.816ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12222024-11-15T20:11:26.816ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12232024-11-15T20:11:26.816ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12242024-11-15T20:11:26.816ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12252024-11-15T20:11:26.817ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12262024-11-15T20:11:26.817ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43385 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12272024-11-15T20:11:26.817ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43385 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12282024-11-15T20:11:26.817ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43385 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12292024-11-15T20:11:26.817ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43385 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12302024-11-15T20:11:26.834ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12312024-11-15T20:11:26.836ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12322024-11-15T20:11:26.836ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12332024-11-15T20:11:26.836ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12342024-11-15T20:11:26.836ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12352024-11-15T20:11:26.836ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12362024-11-15T20:11:26.837ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12372024-11-15T20:11:26.837ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12382024-11-15T20:11:26.837ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40085 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12392024-11-15T20:11:26.837ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:40085 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12402024-11-15T20:11:26.837ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:40085 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12412024-11-15T20:11:26.838ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:40085 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12422024-11-15T20:11:26.839ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12432024-11-15T20:11:26.840ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12442024-11-15T20:11:26.840ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12452024-11-15T20:11:26.840ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12462024-11-15T20:11:26.841ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12472024-11-15T20:11:26.848ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12482024-11-15T20:11:26.848ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12492024-11-15T20:11:26.849ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12502024-11-15T20:11:26.849ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:39545 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12512024-11-15T20:11:26.849ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:39545 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12522024-11-15T20:11:26.849ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:39545 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12532024-11-15T20:11:26.849ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:39545 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12542024-11-15T20:11:26.850ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12552024-11-15T20:11:26.850ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12562024-11-15T20:11:26.850ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12572024-11-15T20:11:26.850ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12582024-11-15T20:11:26.851ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12592024-11-15T20:11:26.851ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12602024-11-15T20:11:26.851ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12612024-11-15T20:11:26.851ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12622024-11-15T20:11:26.851ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43845 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12632024-11-15T20:11:26.852ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43845 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12642024-11-15T20:11:26.852ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43845 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12652024-11-15T20:11:26.852ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43845 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12662024-11-15T20:11:26.853ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12672024-11-15T20:11:26.854ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12682024-11-15T20:11:26.855ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12692024-11-15T20:11:26.855ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12702024-11-15T20:11:26.855ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12712024-11-15T20:11:26.855ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12722024-11-15T20:11:26.855ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12732024-11-15T20:11:26.855ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12742024-11-15T20:11:26.855ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46463 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12752024-11-15T20:11:26.855ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:46463 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12762024-11-15T20:11:26.855ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:46463 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12772024-11-15T20:11:26.856ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:46463 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12782024-11-15T20:11:26.857ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12792024-11-15T20:11:26.857ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12802024-11-15T20:11:26.857ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12812024-11-15T20:11:26.858ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12822024-11-15T20:11:26.858ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12832024-11-15T20:11:26.858ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12842024-11-15T20:11:26.858ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12852024-11-15T20:11:26.858ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12862024-11-15T20:11:26.858ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41589 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12872024-11-15T20:11:26.858ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:41589 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12882024-11-15T20:11:26.858ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:41589 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12892024-11-15T20:11:26.858ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:41589 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12902024-11-15T20:11:26.860ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12912024-11-15T20:11:26.860ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12922024-11-15T20:11:26.861ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12932024-11-15T20:11:26.861ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12942024-11-15T20:11:26.861ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12952024-11-15T20:11:26.861ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12962024-11-15T20:11:26.861ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12972024-11-15T20:11:26.861ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12982024-11-15T20:11:26.862ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45075 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12992024-11-15T20:11:26.862ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45075 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13002024-11-15T20:11:26.863ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45075 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13012024-11-15T20:11:26.863ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45075 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13022024-11-15T20:11:26.864ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13032024-11-15T20:11:26.864ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13042024-11-15T20:11:26.864ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13052024-11-15T20:11:26.864ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13062024-11-15T20:11:26.864ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13072024-11-15T20:11:26.864ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13082024-11-15T20:11:26.865ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13092024-11-15T20:11:26.865ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13102024-11-15T20:11:26.865ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37575 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13112024-11-15T20:11:26.865ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:37575 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13122024-11-15T20:11:26.865ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:37575 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13132024-11-15T20:11:26.865ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:37575 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13142024-11-15T20:11:26.866ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13152024-11-15T20:11:26.868ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13162024-11-15T20:11:26.868ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13172024-11-15T20:11:26.868ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13182024-11-15T20:11:26.869ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13192024-11-15T20:11:26.869ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13202024-11-15T20:11:26.869ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13212024-11-15T20:11:26.869ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13222024-11-15T20:11:26.869ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:43361 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13232024-11-15T20:11:26.869ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:43361 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13242024-11-15T20:11:26.869ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:43361 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13252024-11-15T20:11:26.869ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:43361 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13262024-11-15T20:11:26.870ZDEBGtest_mgs_metrics (init_with_steps): Running step start_crucible_pantry
13272024-11-15T20:11:26.870ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = GET path = /crucible/pantry/0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13282024-11-15T20:11:26.872ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = GET path = /crucible/pantry/0/job/{id}/is-finished sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13292024-11-15T20:11:26.872ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = GET path = /crucible/pantry/0/job/{id}/ok sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13302024-11-15T20:11:26.872ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = DELETE path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13312024-11-15T20:11:26.872ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = GET path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13322024-11-15T20:11:26.872ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = POST path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13332024-11-15T20:11:26.872ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = POST path = /crucible/pantry/0/volume/{id}/background sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13342024-11-15T20:11:26.873ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = POST path = /crucible/pantry/0/volume/{id}/bulk-write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13352024-11-15T20:11:26.873ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = POST path = /crucible/pantry/0/volume/{id}/import-from-url sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13362024-11-15T20:11:26.873ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = POST path = /crucible/pantry/0/volume/{id}/scrub sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13372024-11-15T20:11:26.873ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39023 method = POST path = /crucible/pantry/0/volume/{id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
13382024-11-15T20:11:26.873ZINFOtest_mgs_metrics (dropshot): listening kind = pantry local_addr = [::1]:39023 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13392024-11-15T20:11:26.873ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes kind = pantry local_addr = [::1]:39023 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13402024-11-15T20:11:26.873ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): Started Simulated Crucible Pantry address = [::1]:39023 kind = pantry sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13412024-11-15T20:11:26.873ZDEBGtest_mgs_metrics (init_with_steps): Running step populate_internal_dns
13422024-11-15T20:11:26.873ZDEBGtest_mgs_metrics: Populating Internal DNS
13432024-11-15T20:11:26.941ZINFOtest_mgs_metrics: DNS population: DnsConfigParams { generation: Generation( 1, ), time_created: 2024-11-15T20:11:26.940350587Z, zones: [ DnsConfigZone { zone_name: "control-plane.oxide.internal", records: { "_crucible-pantry._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 39023, target: "c267c503-eaa6-44d1-b2b6-d6c4caa26255.host.control-plane.oxide.internal", }, ), ], "_cockroach._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 46169, target: "37f34d5d-8f1f-4586-9ee7-3774630b5f54.host.control-plane.oxide.internal", }, ), ], "_clickhouse-native._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "baee1955-a5eb-4e52-bcc3-2d4da1309f70.host.control-plane.oxide.internal", }, ), ], "_nexus._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 37069, target: "c74391b6-2721-445b-b829-94c88c82a5d0.host.control-plane.oxide.internal", }, ), ], "b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host": [ Aaaa( ::1, ), ], "_mgd._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 41187, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 43149, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "baee1955-a5eb-4e52-bcc3-2d4da1309f70.host": [ Aaaa( ::1, ), ], "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host": [ Aaaa( ::1, ), ], "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host": [ Aaaa( ::1, ), ], "_mgs._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 45617, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 35733, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_nameservice._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 36837, target: "4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal", }, ), ], "_dendrite._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 43205, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 36977, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "4e890a46-5d01-4c0c-818c-49696ea1d7aa.host": [ Aaaa( ::1, ), ], "c267c503-eaa6-44d1-b2b6-d6c4caa26255.host": [ Aaaa( ::1, ), ], "_external-dns._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 40887, target: "b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal", }, ), ], "c74391b6-2721-445b-b829-94c88c82a5d0.host": [ Aaaa( ::ffff:127.0.0.1, ), ], "_clickhouse._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 46135, target: "baee1955-a5eb-4e52-bcc3-2d4da1309f70.host.control-plane.oxide.internal", }, ), ], "37f34d5d-8f1f-4586-9ee7-3774630b5f54.host": [ Aaaa( ::1, ), ], }, }, ], }
13442024-11-15T20:11:26.941ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:36837/config
13452024-11-15T20:11:27.056ZINFOtest_mgs_metrics (BackgroundTasks): VPC route manager sled 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager
13462024-11-15T20:11:27.056ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:39103/vpc-routes
13472024-11-15T20:11:27.058ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13482024-11-15T20:11:27.058ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13492024-11-15T20:11:27.059ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13502024-11-15T20:11:27.060ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13512024-11-15T20:11:27.061ZTRACtest_mgs_metrics (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = start saga failed total_found = 0
13522024-11-15T20:11:27.061ZWARNtest_mgs_metrics (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
13532024-11-15T20:11:27.061ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 1.528604637s iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
13542024-11-15T20:11:27.062ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
13552024-11-15T20:11:27.062ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
13562024-11-15T20:11:27.062ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
13572024-11-15T20:11:27.063ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13582024-11-15T20:11:27.063ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
13592024-11-15T20:11:27.063ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13602024-11-15T20:11:27.078ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13612024-11-15T20:11:27.079ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 1.540858816s iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
13622024-11-15T20:11:27.079ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:36837 remote_addr = [::1]:44302
13632024-11-15T20:11:27.079ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:39103 remote_addr = [::1]:51750 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13642024-11-15T20:11:27.083ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:39103 method = GET remote_addr = [::1]:51750 req_id = a123d72f-cc78-4dc7-90ad-b9610e798682 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /vpc-routes
13652024-11-15T20:11:27.088ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: f3 bf 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51553 req_id = f9b4a621-baa5-4c56-b82f-4d577232c519
13662024-11-15T20:11:27.088ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62399,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51553 req_id = f9b4a621-baa5-4c56-b82f-4d577232c519
13672024-11-15T20:11:27.088ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62399, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51553 req_id = f9b4a621-baa5-4c56-b82f-4d577232c519
13682024-11-15T20:11:27.089ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 6572 local_addr = [::1]:39103 method = GET remote_addr = [::1]:51750 req_id = a123d72f-cc78-4dc7-90ad-b9610e798682 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /vpc-routes
13692024-11-15T20:11:27.090ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13702024-11-15T20:11:27.100ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13712024-11-15T20:11:27.100ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
13722024-11-15T20:11:27.100ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13732024-11-15T20:11:27.100ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
13742024-11-15T20:11:27.100ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13752024-11-15T20:11:27.103ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13762024-11-15T20:11:27.105ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:36837 method = PUT remote_addr = [::1]:44302 req_id = b8490e67-81f2-4f26-931e-58073ca88715 uri = /config
13772024-11-15T20:11:27.106ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
13782024-11-15T20:11:27.106ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 335.740556ms iteration = 3 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
13792024-11-15T20:11:27.107ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 1 req_id = b8490e67-81f2-4f26-931e-58073ca88715
13802024-11-15T20:11:27.108ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
13812024-11-15T20:11:27.108ZDEBGtest_mgs_metrics (store): creating tree tree_name = generation_1_zone_control-plane.oxide.internal
13822024-11-15T20:11:27.109ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39103/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a123d72f-cc78-4dc7-90ad-b9610e798682", "content-length": "2", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
13832024-11-15T20:11:27.110ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
13842024-11-15T20:11:27.110ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13852024-11-15T20:11:27.112ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13862024-11-15T20:11:27.113ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13872024-11-15T20:11:27.122ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13882024-11-15T20:11:27.122ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
13892024-11-15T20:11:27.123ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13902024-11-15T20:11:27.124ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13912024-11-15T20:11:27.126ZDEBGtest_mgs_metrics (store): updating current config
13922024-11-15T20:11:27.126ZDEBGtest_mgs_metrics (store): flushing default tree
13932024-11-15T20:11:27.136ZINFOtest_mgs_metrics (store): pruning trees for generations older than 1 keep = 3
13942024-11-15T20:11:27.137ZINFOtest_mgs_metrics (store): updated generation new_generation = 1 req_id = b8490e67-81f2-4f26-931e-58073ca88715
13952024-11-15T20:11:27.137ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 32061 local_addr = [::1]:36837 method = PUT remote_addr = [::1]:44302 req_id = b8490e67-81f2-4f26-931e-58073ca88715 response_code = 204 uri = /config
13962024-11-15T20:11:27.137ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13972024-11-15T20:11:27.146ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13982024-11-15T20:11:27.146ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13992024-11-15T20:11:27.146ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14002024-11-15T20:11:27.147ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14012024-11-15T20:11:27.154ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14022024-11-15T20:11:27.154ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14032024-11-15T20:11:27.155ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14042024-11-15T20:11:27.156ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:36837/config", status: 204, headers: {"x-request-id": "b8490e67-81f2-4f26-931e-58073ca88715", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14052024-11-15T20:11:27.157ZDEBGtest_mgs_metrics (init_with_steps): Running step configure_sled_agent1
14062024-11-15T20:11:27.208ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:45047/omicron-zones
14072024-11-15T20:11:27.215ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45047 remote_addr = [::1]:59852 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14082024-11-15T20:11:27.240ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14092024-11-15T20:11:27.240ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14102024-11-15T20:11:27.240ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14112024-11-15T20:11:27.249ZINFOtest_mgs_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-039be560
14122024-11-15T20:11:27.249ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:39103/eip-gateways
14132024-11-15T20:11:27.249ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14142024-11-15T20:11:27.249ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14152024-11-15T20:11:27.251ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14162024-11-15T20:11:27.251ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59852 req_id = 00a40100-9a03-42a6-a329-d5e06e1f6ca3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-zones
14172024-11-15T20:11:27.252ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:39103 method = PUT remote_addr = [::1]:51750 req_id = fd33df0b-ec6c-41c0-9e1d-9233d96877ba sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /eip-gateways
14182024-11-15T20:11:27.254ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 2859 local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59852 req_id = 00a40100-9a03-42a6-a329-d5e06e1f6ca3 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-zones
14192024-11-15T20:11:27.254ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 2499 local_addr = [::1]:39103 method = PUT remote_addr = [::1]:51750 req_id = fd33df0b-ec6c-41c0-9e1d-9233d96877ba response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /eip-gateways
14202024-11-15T20:11:27.259ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39103/eip-gateways", status: 204, headers: {"x-request-id": "fd33df0b-ec6c-41c0-9e1d-9233d96877ba", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14212024-11-15T20:11:27.261ZINFOtest_mgs_metrics (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
14222024-11-15T20:11:27.261ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:45047/vpc-routes
14232024-11-15T20:11:27.261ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
14242024-11-15T20:11:27.261ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14252024-11-15T20:11:27.261ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14262024-11-15T20:11:27.261ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14272024-11-15T20:11:27.262ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14282024-11-15T20:11:27.414ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager body = None method = GET uri = http://[::1]:39103/v2p
14292024-11-15T20:11:27.414ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14302024-11-15T20:11:27.414ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14312024-11-15T20:11:27.414ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14322024-11-15T20:11:27.415ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14332024-11-15T20:11:27.427ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:45047/omicron-zones", status: 204, headers: {"x-request-id": "00a40100-9a03-42a6-a329-d5e06e1f6ca3", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14342024-11-15T20:11:27.428ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:45047/network-bootstore-config
14352024-11-15T20:11:27.428ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45047 remote_addr = [::1]:59854 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14362024-11-15T20:11:27.428ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:39103 remote_addr = [::1]:51752 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
14372024-11-15T20:11:27.430ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45047 method = GET remote_addr = [::1]:59854 req_id = 1d778d7c-5325-477c-b655-7b0d31133ce6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
14382024-11-15T20:11:27.432ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:39103 method = GET remote_addr = [::1]:51752 req_id = 0d4daeb0-fa18-4688-b783-94e9ea78fbdf sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
14392024-11-15T20:11:27.432ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59852 req_id = 4695cbd8-db46-4f4b-beec-e08c67b6d38d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
14402024-11-15T20:11:27.438ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 7108 local_addr = [::1]:45047 method = GET remote_addr = [::1]:59854 req_id = 1d778d7c-5325-477c-b655-7b0d31133ce6 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
14412024-11-15T20:11:27.438ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 6114 local_addr = [::1]:39103 method = GET remote_addr = [::1]:51752 req_id = 0d4daeb0-fa18-4688-b783-94e9ea78fbdf response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
14422024-11-15T20:11:27.438ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 6145 local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59852 req_id = 4695cbd8-db46-4f4b-beec-e08c67b6d38d response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
14432024-11-15T20:11:27.440ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14442024-11-15T20:11:27.440ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14452024-11-15T20:11:27.440ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14462024-11-15T20:11:27.440ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14472024-11-15T20:11:27.456ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:45047/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d778d7c-5325-477c-b655-7b0d31133ce6", "content-length": "2", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14482024-11-15T20:11:27.456ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
14492024-11-15T20:11:27.456ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14502024-11-15T20:11:27.457ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14512024-11-15T20:11:27.457ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:39103/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d4daeb0-fa18-4688-b783-94e9ea78fbdf", "content-length": "2", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14522024-11-15T20:11:27.457ZINFOtest_mgs_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-039be560
14532024-11-15T20:11:27.457ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-039be560
14542024-11-15T20:11:27.457ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-039be560
14552024-11-15T20:11:27.458ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:45047/v2p
14562024-11-15T20:11:27.459ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14572024-11-15T20:11:27.471ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
14582024-11-15T20:11:27.471ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
14592024-11-15T20:11:27.471ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14602024-11-15T20:11:27.471ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14612024-11-15T20:11:27.473ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14622024-11-15T20:11:27.473ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:45047/network-bootstore-config", status: 204, headers: {"x-request-id": "4695cbd8-db46-4f4b-beec-e08c67b6d38d", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14632024-11-15T20:11:27.473ZDEBGtest_mgs_metrics (init_with_steps): Running step configure_sled_agent2
14642024-11-15T20:11:27.570ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:39103/omicron-zones
14652024-11-15T20:11:27.571ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45047 remote_addr = [::1]:59856 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14662024-11-15T20:11:27.574ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:39103 remote_addr = [::1]:51754 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
14672024-11-15T20:11:27.574ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45047 method = GET remote_addr = [::1]:59856 req_id = a8223e7d-85e1-48b6-8e47-8bdbd3bb0066 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
14682024-11-15T20:11:27.575ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 1001 local_addr = [::1]:45047 method = GET remote_addr = [::1]:59856 req_id = a8223e7d-85e1-48b6-8e47-8bdbd3bb0066 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
14692024-11-15T20:11:27.583ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:45047/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8223e7d-85e1-48b6-8e47-8bdbd3bb0066", "content-length": "2", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14702024-11-15T20:11:27.583ZINFOtest_mgs_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
14712024-11-15T20:11:27.583ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
14722024-11-15T20:11:27.583ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
14732024-11-15T20:11:27.585ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 2.055697289s iteration = 1 status = {}
14742024-11-15T20:11:27.585ZINFOtest_mgs_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
14752024-11-15T20:11:27.586ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:45047/eip-gateways
14762024-11-15T20:11:27.586ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14772024-11-15T20:11:27.586ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14782024-11-15T20:11:27.589ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14792024-11-15T20:11:27.590ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:39103 method = PUT remote_addr = [::1]:51754 req_id = a97df622-d718-4ae6-afc1-2e1c36267945 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /omicron-zones
14802024-11-15T20:11:27.590ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59854 req_id = 51666a5f-edc2-4e71-bc01-13105dc063bc sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
14812024-11-15T20:11:27.591ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 862 local_addr = [::1]:39103 method = PUT remote_addr = [::1]:51754 req_id = a97df622-d718-4ae6-afc1-2e1c36267945 response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /omicron-zones
14822024-11-15T20:11:27.591ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 477 local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59854 req_id = 51666a5f-edc2-4e71-bc01-13105dc063bc response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
14832024-11-15T20:11:27.596ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:45047/eip-gateways", status: 204, headers: {"x-request-id": "51666a5f-edc2-4e71-bc01-13105dc063bc", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14842024-11-15T20:11:27.597ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 2.063673565s iteration = 1 status = {}
14852024-11-15T20:11:27.598ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14862024-11-15T20:11:27.599ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14872024-11-15T20:11:27.599ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14882024-11-15T20:11:27.600ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:39103/omicron-zones", status: 204, headers: {"x-request-id": "a97df622-d718-4ae6-afc1-2e1c36267945", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14892024-11-15T20:11:27.601ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:39103/network-bootstore-config
14902024-11-15T20:11:27.601ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:39103 method = PUT remote_addr = [::1]:51754 req_id = c3d93b96-cdab-483a-854a-a5b440c088fe sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /network-bootstore-config
14912024-11-15T20:11:27.603ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 1923 local_addr = [::1]:39103 method = PUT remote_addr = [::1]:51754 req_id = c3d93b96-cdab-483a-854a-a5b440c088fe response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /network-bootstore-config
14922024-11-15T20:11:27.603ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:39103/network-bootstore-config", status: 204, headers: {"x-request-id": "c3d93b96-cdab-483a-854a-a5b440c088fe", "date": "Fri, 15 Nov 2024 20:11:26 GMT"} })
14932024-11-15T20:11:27.607ZDEBGtest_mgs_metrics (init_with_steps): Running step start_nexus_external
14942024-11-15T20:11:27.608ZDEBGtest_mgs_metrics: Starting Nexus (external API)
14952024-11-15T20:11:28.992ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
14962024-11-15T20:11:28.992ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14972024-11-15T20:11:28.992ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14982024-11-15T20:11:28.992ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
14992024-11-15T20:11:28.993ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15002024-11-15T20:11:28.994ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15012024-11-15T20:11:28.995ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15022024-11-15T20:11:28.995ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15032024-11-15T20:11:28.999ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
15042024-11-15T20:11:29.000ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15052024-11-15T20:11:29.000ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15062024-11-15T20:11:29.000ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15072024-11-15T20:11:29.001ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15082024-11-15T20:11:29.011ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15092024-11-15T20:11:29.011ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15102024-11-15T20:11:29.011ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15112024-11-15T20:11:29.013ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15122024-11-15T20:11:29.015ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15132024-11-15T20:11:29.015ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15142024-11-15T20:11:29.015ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15152024-11-15T20:11:29.015ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15162024-11-15T20:11:29.020ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15172024-11-15T20:11:29.027ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15182024-11-15T20:11:29.027ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15192024-11-15T20:11:29.027ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15202024-11-15T20:11:29.028ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15212024-11-15T20:11:29.035ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15222024-11-15T20:11:29.048ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
15232024-11-15T20:11:29.048ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
15242024-11-15T20:11:29.048ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
15252024-11-15T20:11:29.048ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15262024-11-15T20:11:29.049ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15272024-11-15T20:11:29.049ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15282024-11-15T20:11:29.049ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15292024-11-15T20:11:29.059ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15302024-11-15T20:11:29.059ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15312024-11-15T20:11:29.059ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15322024-11-15T20:11:29.065ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15332024-11-15T20:11:29.065ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15342024-11-15T20:11:29.065ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15352024-11-15T20:11:29.065ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15362024-11-15T20:11:29.078ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15372024-11-15T20:11:29.078ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15382024-11-15T20:11:29.078ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15392024-11-15T20:11:29.078ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15402024-11-15T20:11:29.081ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15412024-11-15T20:11:29.090ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
15422024-11-15T20:11:29.090ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15432024-11-15T20:11:29.090ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15442024-11-15T20:11:29.091ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15452024-11-15T20:11:29.103ZINFOtest_mgs_metrics (InternalApi): set cockroachdb setting setting = cluster.preserve_downgrade_option value = 22.1
15462024-11-15T20:11:29.103ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15472024-11-15T20:11:29.103ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15482024-11-15T20:11:29.105ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15492024-11-15T20:11:29.111ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15502024-11-15T20:11:29.111ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15512024-11-15T20:11:29.112ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15522024-11-15T20:11:29.132ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15532024-11-15T20:11:29.132ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15542024-11-15T20:11:29.133ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15552024-11-15T20:11:29.134ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
15562024-11-15T20:11:29.134ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15572024-11-15T20:11:29.134ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15582024-11-15T20:11:29.134ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15592024-11-15T20:11:29.136ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15602024-11-15T20:11:29.139ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15612024-11-15T20:11:29.140ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15622024-11-15T20:11:29.140ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15632024-11-15T20:11:29.150ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15642024-11-15T20:11:29.162ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
15652024-11-15T20:11:29.162ZDEBGtest_mgs_metrics (InternalApi): updating rack subnet for rack c19a698f-c6f9-4a17-ae30-20d711b8f7dc to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), )
15662024-11-15T20:11:29.163ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15672024-11-15T20:11:29.163ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15682024-11-15T20:11:29.163ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15692024-11-15T20:11:29.165ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15702024-11-15T20:11:29.165ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15712024-11-15T20:11:29.165ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15722024-11-15T20:11:29.183ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
15732024-11-15T20:11:29.183ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15742024-11-15T20:11:29.183ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15752024-11-15T20:11:29.183ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15762024-11-15T20:11:29.186ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15772024-11-15T20:11:29.187ZINFOtest_mgs_metrics (InternalApi): Using static configuration for external switchports
15782024-11-15T20:11:29.187ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15792024-11-15T20:11:29.187ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15802024-11-15T20:11:29.188ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15812024-11-15T20:11:29.189ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15822024-11-15T20:11:29.190ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15832024-11-15T20:11:29.190ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15842024-11-15T20:11:29.190ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15852024-11-15T20:11:29.191ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15862024-11-15T20:11:29.191ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15872024-11-15T20:11:29.191ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15882024-11-15T20:11:29.192ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15892024-11-15T20:11:29.205ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15902024-11-15T20:11:29.219ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15912024-11-15T20:11:29.219ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
15922024-11-15T20:11:29.219ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
15932024-11-15T20:11:29.219ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15942024-11-15T20:11:29.219ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15952024-11-15T20:11:29.219ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15962024-11-15T20:11:29.220ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15972024-11-15T20:11:29.223ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15982024-11-15T20:11:29.224ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15992024-11-15T20:11:29.224ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16002024-11-15T20:11:29.224ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16012024-11-15T20:11:29.230ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16022024-11-15T20:11:29.230ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16032024-11-15T20:11:29.231ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16042024-11-15T20:11:29.231ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16052024-11-15T20:11:29.250ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16062024-11-15T20:11:29.268ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
16072024-11-15T20:11:29.269ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16082024-11-15T20:11:29.269ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16092024-11-15T20:11:29.269ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16102024-11-15T20:11:29.285ZINFOtest_mgs_metrics (InternalApi): Recording Rack Network Configuration
16112024-11-15T20:11:29.286ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16122024-11-15T20:11:29.286ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16132024-11-15T20:11:29.286ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16142024-11-15T20:11:29.329ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
16152024-11-15T20:11:29.330ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet
16162024-11-15T20:11:29.330ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16172024-11-15T20:11:29.331ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16182024-11-15T20:11:29.331ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16192024-11-15T20:11:29.331ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16202024-11-15T20:11:29.352ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16212024-11-15T20:11:29.360ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
16222024-11-15T20:11:29.360ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16232024-11-15T20:11:29.360ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16242024-11-15T20:11:29.360ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16252024-11-15T20:11:29.370ZTRACtest_mgs_metrics (service-firewall-rule-progator): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-11-15T19:50:02.602910Z, time_modified: 2024-11-15T19:50:02.602910Z, allowed_ips: None } background_task = service_firewall_rule_propagation
16262024-11-15T20:11:29.370ZDEBGtest_mgs_metrics (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
16272024-11-15T20:11:29.370ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
16282024-11-15T20:11:29.370ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
16292024-11-15T20:11:29.533ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16302024-11-15T20:11:29.534ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16312024-11-15T20:11:29.534ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16322024-11-15T20:11:29.534ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16332024-11-15T20:11:29.534ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16342024-11-15T20:11:29.537ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16352024-11-15T20:11:29.541ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16362024-11-15T20:11:29.541ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16372024-11-15T20:11:29.541ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16382024-11-15T20:11:29.542ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16392024-11-15T20:11:29.544ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 94a50be8-8657-4e24-860e-7b808627ab3d, lookup_type: ByName("oxide-service-pool") }
16402024-11-15T20:11:29.545ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16412024-11-15T20:11:29.545ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16422024-11-15T20:11:29.545ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16432024-11-15T20:11:29.545ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16442024-11-15T20:11:29.548ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16452024-11-15T20:11:29.562ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 94a50be8-8657-4e24-860e-7b808627ab3d, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16462024-11-15T20:11:29.565ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16472024-11-15T20:11:29.567ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16482024-11-15T20:11:29.567ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16492024-11-15T20:11:29.571ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 94a50be8-8657-4e24-860e-7b808627ab3d, lookup_type: ByName("oxide-service-pool") }
16502024-11-15T20:11:29.571ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16512024-11-15T20:11:29.572ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16522024-11-15T20:11:29.573ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16532024-11-15T20:11:29.573ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16542024-11-15T20:11:29.576ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16552024-11-15T20:11:29.588ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 94a50be8-8657-4e24-860e-7b808627ab3d, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16562024-11-15T20:11:29.592ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
16572024-11-15T20:11:29.593ZDEBGtest_mgs_metrics (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
16582024-11-15T20:11:29.593ZINFOtest_mgs_metrics (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation
16592024-11-15T20:11:29.593ZDEBGtest_mgs_metrics (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 4.059226037s
16602024-11-15T20:11:29.593ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 4.05926545s iteration = 1 status = {}
16612024-11-15T20:11:29.600ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 94a50be8-8657-4e24-860e-7b808627ab3d, lookup_type: ByName("oxide-service-pool") }
16622024-11-15T20:11:29.601ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16632024-11-15T20:11:29.601ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16642024-11-15T20:11:29.601ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16652024-11-15T20:11:29.602ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16662024-11-15T20:11:29.604ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16672024-11-15T20:11:29.609ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 94a50be8-8657-4e24-860e-7b808627ab3d, lookup_type: ByName("oxide-service-pool") } result = Ok(())
16682024-11-15T20:11:29.614ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
16692024-11-15T20:11:29.614ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16702024-11-15T20:11:29.615ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16712024-11-15T20:11:29.615ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16722024-11-15T20:11:29.616ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16732024-11-15T20:11:29.618ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16742024-11-15T20:11:29.624ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
16752024-11-15T20:11:29.922ZINFOtest_mgs_metrics (InternalApi): inserted blueprint blueprint_id = 07ec69b2-85b9-426f-968f-582c7bd48539
16762024-11-15T20:11:29.922ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
16772024-11-15T20:11:29.922ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16782024-11-15T20:11:29.922ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16792024-11-15T20:11:29.922ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16802024-11-15T20:11:29.922ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16812024-11-15T20:11:29.922ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16822024-11-15T20:11:29.925ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
16832024-11-15T20:11:29.933ZINFOtest_mgs_metrics (InternalApi): No networking records needed for cockroach_db service
16842024-11-15T20:11:29.934ZINFOtest_mgs_metrics (InternalApi): No networking records needed for clickhouse service
16852024-11-15T20:11:29.934ZINFOtest_mgs_metrics (InternalApi): No networking records needed for internal_dns service
16862024-11-15T20:11:30.021ZINFOtest_mgs_metrics (InternalApi): Inserted networking records for external_dns service
16872024-11-15T20:11:30.062ZINFOtest_mgs_metrics (InternalApi): Inserted networking records for nexus service
16882024-11-15T20:11:30.063ZINFOtest_mgs_metrics (InternalApi): No networking records needed for crucible_pantry service
16892024-11-15T20:11:30.063ZINFOtest_mgs_metrics (InternalApi): Inserted service networking records
16902024-11-15T20:11:30.063ZINFOtest_mgs_metrics (InternalApi): Inserted physical disks
16912024-11-15T20:11:30.063ZINFOtest_mgs_metrics (InternalApi): Inserted zpools
16922024-11-15T20:11:30.063ZINFOtest_mgs_metrics (InternalApi): Inserted datasets
16932024-11-15T20:11:30.086ZINFOtest_mgs_metrics (InternalApi): Populated DNS tables for internal DNS
16942024-11-15T20:11:30.092ZINFOtest_mgs_metrics (InternalApi): Populated DNS tables for external DNS
16952024-11-15T20:11:30.092ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16962024-11-15T20:11:30.092ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16972024-11-15T20:11:30.092ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16982024-11-15T20:11:30.092ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16992024-11-15T20:11:30.092ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17002024-11-15T20:11:30.092ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17012024-11-15T20:11:30.098ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17022024-11-15T20:11:30.102ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
17032024-11-15T20:11:30.102ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17042024-11-15T20:11:30.102ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17052024-11-15T20:11:30.102ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17062024-11-15T20:11:30.102ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17072024-11-15T20:11:30.105ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17082024-11-15T20:11:30.113ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17092024-11-15T20:11:30.160ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
17102024-11-15T20:11:30.160ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17112024-11-15T20:11:30.160ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17122024-11-15T20:11:30.160ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17132024-11-15T20:11:30.161ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17142024-11-15T20:11:30.163ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17152024-11-15T20:11:30.177ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
17162024-11-15T20:11:30.178ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
17172024-11-15T20:11:30.178ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17182024-11-15T20:11:30.178ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17192024-11-15T20:11:30.178ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17202024-11-15T20:11:30.178ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17212024-11-15T20:11:30.181ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17222024-11-15T20:11:30.206ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
17232024-11-15T20:11:30.211ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
17242024-11-15T20:11:30.211ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17252024-11-15T20:11:30.211ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17262024-11-15T20:11:30.211ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17272024-11-15T20:11:30.213ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17282024-11-15T20:11:30.215ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17292024-11-15T20:11:30.225ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
17302024-11-15T20:11:30.240ZINFOtest_mgs_metrics (InternalApi): Created recovery silo
17312024-11-15T20:11:30.241ZINFOtest_mgs_metrics (InternalApi): Created recovery user
17322024-11-15T20:11:30.251ZINFOtest_mgs_metrics (InternalApi): Created recovery user's password
17332024-11-15T20:11:30.252ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: d6d76869-d8e8-42dd-92fb-71e6ce54721e, lookup_type: ById(d6d76869-d8e8-42dd-92fb-71e6ce54721e) }
17342024-11-15T20:11:30.252ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = d6d76869-d8e8-42dd-92fb-71e6ce54721e resource_type = Silo
17352024-11-15T20:11:30.252ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17362024-11-15T20:11:30.252ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17372024-11-15T20:11:30.255ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17382024-11-15T20:11:30.259ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17392024-11-15T20:11:30.260ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17402024-11-15T20:11:30.260ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17412024-11-15T20:11:30.260ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17422024-11-15T20:11:30.262ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17432024-11-15T20:11:30.266ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: d6d76869-d8e8-42dd-92fb-71e6ce54721e, lookup_type: ById(d6d76869-d8e8-42dd-92fb-71e6ce54721e) } result = Ok(())
17442024-11-15T20:11:30.266ZDEBGtest_mgs_metrics (InternalApi): Generated role assignment queries
17452024-11-15T20:11:30.270ZINFOtest_mgs_metrics (InternalApi): Granted Silo privileges
17462024-11-15T20:11:30.271ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
17472024-11-15T20:11:30.271ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17482024-11-15T20:11:30.271ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17492024-11-15T20:11:30.271ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17502024-11-15T20:11:30.271ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17512024-11-15T20:11:30.276ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17522024-11-15T20:11:30.279ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17532024-11-15T20:11:30.306ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17542024-11-15T20:11:30.306ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17552024-11-15T20:11:30.309ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17562024-11-15T20:11:30.312ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17572024-11-15T20:11:30.312ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17582024-11-15T20:11:30.312ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17592024-11-15T20:11:30.313ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17602024-11-15T20:11:30.314ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17612024-11-15T20:11:30.314ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17622024-11-15T20:11:30.316ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17632024-11-15T20:11:30.316ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17642024-11-15T20:11:30.316ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17652024-11-15T20:11:30.316ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17662024-11-15T20:11:30.316ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17672024-11-15T20:11:30.323ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17682024-11-15T20:11:30.323ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17692024-11-15T20:11:30.323ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17702024-11-15T20:11:30.324ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17712024-11-15T20:11:30.329ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17722024-11-15T20:11:30.330ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17732024-11-15T20:11:30.330ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17742024-11-15T20:11:30.330ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17752024-11-15T20:11:30.332ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17762024-11-15T20:11:30.343ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17772024-11-15T20:11:30.344ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
17782024-11-15T20:11:30.344ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17792024-11-15T20:11:30.345ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17802024-11-15T20:11:30.345ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17812024-11-15T20:11:30.345ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17822024-11-15T20:11:30.345ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17832024-11-15T20:11:30.345ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17842024-11-15T20:11:30.346ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17852024-11-15T20:11:30.346ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17862024-11-15T20:11:30.347ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17872024-11-15T20:11:30.349ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17882024-11-15T20:11:30.349ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17892024-11-15T20:11:30.350ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17902024-11-15T20:11:30.350ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17912024-11-15T20:11:30.354ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17922024-11-15T20:11:30.363ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17932024-11-15T20:11:30.365ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17942024-11-15T20:11:30.365ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17952024-11-15T20:11:30.365ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17962024-11-15T20:11:30.379ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17972024-11-15T20:11:30.379ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17982024-11-15T20:11:30.380ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17992024-11-15T20:11:30.383ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18002024-11-15T20:11:30.384ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18012024-11-15T20:11:30.384ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18022024-11-15T20:11:30.386ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18032024-11-15T20:11:30.390ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18042024-11-15T20:11:30.390ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18052024-11-15T20:11:30.390ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
18062024-11-15T20:11:30.390ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18072024-11-15T20:11:30.390ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18082024-11-15T20:11:30.390ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18092024-11-15T20:11:30.392ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18102024-11-15T20:11:30.396ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18112024-11-15T20:11:30.396ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18122024-11-15T20:11:30.396ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18132024-11-15T20:11:30.399ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18142024-11-15T20:11:30.403ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18152024-11-15T20:11:30.403ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18162024-11-15T20:11:30.403ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18172024-11-15T20:11:30.404ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18182024-11-15T20:11:30.407ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18192024-11-15T20:11:30.416ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
18202024-11-15T20:11:30.416ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
18212024-11-15T20:11:30.416ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18222024-11-15T20:11:30.416ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18232024-11-15T20:11:30.416ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18242024-11-15T20:11:30.417ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18252024-11-15T20:11:30.417ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18262024-11-15T20:11:30.418ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18272024-11-15T20:11:30.422ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18282024-11-15T20:11:30.422ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18292024-11-15T20:11:30.422ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18302024-11-15T20:11:30.425ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18312024-11-15T20:11:30.425ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18322024-11-15T20:11:30.426ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18332024-11-15T20:11:30.432ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
18342024-11-15T20:11:30.433ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18352024-11-15T20:11:30.433ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18362024-11-15T20:11:30.433ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18372024-11-15T20:11:30.433ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18382024-11-15T20:11:30.435ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18392024-11-15T20:11:30.436ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18402024-11-15T20:11:30.436ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18412024-11-15T20:11:30.437ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18422024-11-15T20:11:30.438ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18432024-11-15T20:11:30.439ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18442024-11-15T20:11:30.439ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18452024-11-15T20:11:30.439ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18462024-11-15T20:11:30.442ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18472024-11-15T20:11:30.450ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
18482024-11-15T20:11:30.450ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
18492024-11-15T20:11:30.450ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
18502024-11-15T20:11:30.450ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18512024-11-15T20:11:30.450ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18522024-11-15T20:11:30.450ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18532024-11-15T20:11:30.450ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18542024-11-15T20:11:30.453ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18552024-11-15T20:11:30.453ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18562024-11-15T20:11:30.453ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18572024-11-15T20:11:30.453ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18582024-11-15T20:11:30.456ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18592024-11-15T20:11:30.456ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18602024-11-15T20:11:30.456ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18612024-11-15T20:11:30.463ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18622024-11-15T20:11:30.465ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18632024-11-15T20:11:30.474ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
18642024-11-15T20:11:30.474ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18652024-11-15T20:11:30.474ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18662024-11-15T20:11:30.474ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18672024-11-15T20:11:30.482ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18682024-11-15T20:11:30.482ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18692024-11-15T20:11:30.483ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18702024-11-15T20:11:30.484ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18712024-11-15T20:11:30.485ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18722024-11-15T20:11:30.485ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18732024-11-15T20:11:30.492ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18742024-11-15T20:11:30.494ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18752024-11-15T20:11:30.494ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18762024-11-15T20:11:30.497ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18772024-11-15T20:11:30.499ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18782024-11-15T20:11:30.499ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18792024-11-15T20:11:30.500ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
18802024-11-15T20:11:30.500ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18812024-11-15T20:11:30.500ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18822024-11-15T20:11:30.500ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18832024-11-15T20:11:30.502ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18842024-11-15T20:11:30.504ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18852024-11-15T20:11:30.504ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18862024-11-15T20:11:30.504ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18872024-11-15T20:11:30.505ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18882024-11-15T20:11:30.509ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18892024-11-15T20:11:30.509ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18902024-11-15T20:11:30.509ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18912024-11-15T20:11:30.510ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18922024-11-15T20:11:30.512ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18932024-11-15T20:11:30.527ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18942024-11-15T20:11:30.527ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
18952024-11-15T20:11:30.527ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
18962024-11-15T20:11:30.527ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18972024-11-15T20:11:30.528ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18982024-11-15T20:11:30.528ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18992024-11-15T20:11:30.528ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19002024-11-15T20:11:30.529ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
19012024-11-15T20:11:30.531ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19022024-11-15T20:11:30.531ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19032024-11-15T20:11:30.532ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19042024-11-15T20:11:30.532ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19052024-11-15T20:11:30.532ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19062024-11-15T20:11:30.532ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19072024-11-15T20:11:30.532ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19082024-11-15T20:11:30.536ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19092024-11-15T20:11:30.545ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
19102024-11-15T20:11:30.545ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19112024-11-15T20:11:30.545ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19122024-11-15T20:11:30.546ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19132024-11-15T20:11:30.554ZDEBGtest_mgs_metrics (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 9be58fc7-4cf7-4b1e-94eb-817a0b5a6927, kind: Service { id: c74391b6-2721-445b-b829-94c88c82a5d0 }, name: Name("nexus-c74391b6-2721-445b-b829-94c88c82a5d0"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 7c44a9cf-0a90-4b3d-a2cd-737ce92945f4, kind: Service { id: b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82 }, name: Name("external-dns-b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
19142024-11-15T20:11:30.555ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
19152024-11-15T20:11:30.555ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19162024-11-15T20:11:30.555ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19172024-11-15T20:11:30.555ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19182024-11-15T20:11:30.555ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19192024-11-15T20:11:30.557ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19202024-11-15T20:11:30.562ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
19212024-11-15T20:11:30.563ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19222024-11-15T20:11:30.563ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19232024-11-15T20:11:30.563ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19242024-11-15T20:11:30.566ZTRACtest_mgs_metrics (nexus): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-11-15T19:50:02.602910Z, time_modified: 2024-11-15T20:11:30.279470Z, allowed_ips: None }
19252024-11-15T20:11:30.567ZDEBGtest_mgs_metrics (nexus): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
19262024-11-15T20:11:30.567ZDEBGtest_mgs_metrics (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 7c44a9cf-0a90-4b3d-a2cd-737ce92945f4, kind: Service { id: b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82 }, name: Name("external-dns-b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 9be58fc7-4cf7-4b1e-94eb-817a0b5a6927, kind: Service { id: c74391b6-2721-445b-b829-94c88c82a5d0 }, name: Name("nexus-c74391b6-2721-445b-b829-94c88c82a5d0"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
19272024-11-15T20:11:30.567ZDEBGtest_mgs_metrics (nexus): resolved 2 rules for sleds
19282024-11-15T20:11:30.684ZDEBGtest_mgs_metrics (nexus): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2024-11-15T20:11:25.540956Z, time_modified: 2024-11-15T20:11:25.540956Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(45047), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
19292024-11-15T20:11:30.684ZDEBGtest_mgs_metrics (nexus): sending firewall rules to sled agents
19302024-11-15T20:11:30.684ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
19312024-11-15T20:11:30.685ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {} }
19322024-11-15T20:11:30.686ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
19332024-11-15T20:11:30.690ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
19342024-11-15T20:11:30.690ZTRACtest_mgs_metrics (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19352024-11-15T20:11:30.692ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
19362024-11-15T20:11:30.692ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {} }
19372024-11-15T20:11:30.692ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
19382024-11-15T20:11:30.704ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
19392024-11-15T20:11:30.711ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
19402024-11-15T20:11:30.762ZDEBGtest_mgs_metrics (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:45047/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19412024-11-15T20:11:30.763ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45047 remote_addr = [::1]:59858 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
19422024-11-15T20:11:30.764ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59858 req_id = b048a71c-651a-446f-b4dc-eba0ad851893 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19432024-11-15T20:11:30.764ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 777 local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59858 req_id = b048a71c-651a-446f-b4dc-eba0ad851893 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19442024-11-15T20:11:30.765ZDEBGtest_mgs_metrics (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:45047/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "b048a71c-651a-446f-b4dc-eba0ad851893", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
19452024-11-15T20:11:30.766ZINFOtest_mgs_metrics (nexus): updated firewall rules on 1 sleds
19462024-11-15T20:11:30.766ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
19472024-11-15T20:11:30.767ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {} }
19482024-11-15T20:11:30.767ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
19492024-11-15T20:11:30.767ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
19502024-11-15T20:11:30.767ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19512024-11-15T20:11:30.767ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19522024-11-15T20:11:30.767ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19532024-11-15T20:11:30.768ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19542024-11-15T20:11:30.769ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
19552024-11-15T20:11:30.769ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
19562024-11-15T20:11:30.769ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
19572024-11-15T20:11:30.769ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19582024-11-15T20:11:30.769ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19592024-11-15T20:11:30.769ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19602024-11-15T20:11:30.769ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19612024-11-15T20:11:30.769ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
19622024-11-15T20:11:30.770ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
19632024-11-15T20:11:30.770ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
19642024-11-15T20:11:30.770ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
19652024-11-15T20:11:30.770ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19662024-11-15T20:11:30.770ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19672024-11-15T20:11:30.770ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19682024-11-15T20:11:30.771ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19692024-11-15T20:11:30.771ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Signaled
19702024-11-15T20:11:30.771ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
19712024-11-15T20:11:30.773ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19722024-11-15T20:11:30.773ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
19732024-11-15T20:11:30.773ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
19742024-11-15T20:11:30.773ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
19752024-11-15T20:11:30.773ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: a3 76 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 .v..........._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:64315 req_id = 3c5bed48-7c86-48ba-a87b-802c72cfd9ec
19762024-11-15T20:11:30.773ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41846,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64315 req_id = 3c5bed48-7c86-48ba-a87b-802c72cfd9ec
19772024-11-15T20:11:30.774ZDEBGtest_mgs_metrics (store): query key key = _nameservice._tcp
19782024-11-15T20:11:30.774ZDEBGtest_mgs_metrics (store): query key key = 4e890a46-5d01-4c0c-818c-49696ea1d7aa.host
19792024-11-15T20:11:30.774ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64315 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36837, target: Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal") })) }] req_id = 3c5bed48-7c86-48ba-a87b-802c72cfd9ec
19802024-11-15T20:11:30.774ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 4d 4c 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 ML..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55000 req_id = d660a792-9358-4d82-8cf6-1a81b56f86e5
19812024-11-15T20:11:30.775ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19788,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55000 req_id = d660a792-9358-4d82-8cf6-1a81b56f86e5
19822024-11-15T20:11:30.775ZDEBGtest_mgs_metrics (store): query key key = _external-dns._tcp
19832024-11-15T20:11:30.775ZDEBGtest_mgs_metrics (store): query key key = b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host
19842024-11-15T20:11:30.777ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55000 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40887, target: Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal") })) }] req_id = d660a792-9358-4d82-8cf6-1a81b56f86e5
19852024-11-15T20:11:30.777ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36837, target: Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2040, tv_nsec: 574794859 } })
19862024-11-15T20:11:30.777ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
19872024-11-15T20:11:30.777ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40887, target: Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2040, tv_nsec: 575417833 } })
19882024-11-15T20:11:30.777ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
19892024-11-15T20:11:30.778ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19902024-11-15T20:11:30.778ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19912024-11-15T20:11:30.778ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19922024-11-15T20:11:30.778ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19932024-11-15T20:11:30.779ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19942024-11-15T20:11:30.779ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19952024-11-15T20:11:30.780ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19962024-11-15T20:11:30.780ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19972024-11-15T20:11:30.780ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19982024-11-15T20:11:30.780ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19992024-11-15T20:11:30.780ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
20002024-11-15T20:11:30.781ZTRACtest_mgs_metrics (ServiceBalancer): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20012024-11-15T20:11:30.781ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
20022024-11-15T20:11:30.781ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {} }
20032024-11-15T20:11:30.782ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
20042024-11-15T20:11:30.785ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20052024-11-15T20:11:30.789ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20062024-11-15T20:11:30.789ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20072024-11-15T20:11:30.789ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20082024-11-15T20:11:30.790ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20092024-11-15T20:11:30.796ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20102024-11-15T20:11:30.802ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
20112024-11-15T20:11:30.805ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
20122024-11-15T20:11:30.805ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
20132024-11-15T20:11:30.810ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
20142024-11-15T20:11:30.810ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20152024-11-15T20:11:30.816ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20162024-11-15T20:11:30.818ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20172024-11-15T20:11:30.824ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20182024-11-15T20:11:30.825ZDEBGtest_mgs_metrics (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
20192024-11-15T20:11:30.825ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
20202024-11-15T20:11:30.825ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
20212024-11-15T20:11:30.825ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20222024-11-15T20:11:30.825ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20232024-11-15T20:11:30.831ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20242024-11-15T20:11:30.831ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20252024-11-15T20:11:30.831ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20262024-11-15T20:11:30.831ZDEBGtest_mgs_metrics (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
20272024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
20282024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
20292024-11-15T20:11:30.832ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
20302024-11-15T20:11:30.832ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20312024-11-15T20:11:30.832ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20322024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20332024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20342024-11-15T20:11:30.832ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 87 36 01 00 00 01 00 00 00 00 00 01 24 34 65 38 .6..........$4e8\n0010: 39 30 61 34 36 2d 35 64 30 31 2d 34 63 30 63 2d 90a46-5d01-4c0c-\n0020: 38 31 38 63 2d 34 39 36 39 36 65 61 31 64 37 61 818c-49696ea1d7a\n0030: 61 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 a.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:63714 req_id = ab952c2a-a3a3-404b-b6f9-67ff671b80de
20352024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34614,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 52,\n 101,\n 56,\n 57,\n 48,\n 97,\n 52,\n 54,\n 45,\n 53,\n 100,\n 48,\n 49,\n 45,\n 52,\n 99,\n 48,\n 99,\n 45,\n 56,\n 49,\n 56,\n 99,\n 45,\n 52,\n 57,\n 54,\n 57,\n 54,\n 101,\n 97,\n 49,\n 100,\n 55,\n 97,\n 97,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63714 req_id = ab952c2a-a3a3-404b-b6f9-67ff671b80de
20362024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (store): query key key = 4e890a46-5d01-4c0c-818c-49696ea1d7aa.host
20372024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63714 query = LowerQuery { name: LowerName(Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal.")), original: Query { name: Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("4e890a46-5d01-4c0c-818c-49696ea1d7aa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ab952c2a-a3a3-404b-b6f9-67ff671b80de
20382024-11-15T20:11:30.832ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: a0 ad 01 00 00 01 00 00 00 00 00 01 24 62 37 63 ............$b7c\n0010: 33 64 63 35 35 2d 63 30 61 37 2d 34 38 35 34 2d 3dc55-c0a7-4854-\n0020: 39 62 37 63 2d 61 31 65 61 61 63 31 61 31 61 38 9b7c-a1eaac1a1a8\n0030: 32 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 2.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:60428 req_id = 9bdfd14e-69a9-44ea-99ea-339b5e420141
20392024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41133,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 98,\n 55,\n 99,\n 51,\n 100,\n 99,\n 53,\n 53,\n 45,\n 99,\n 48,\n 97,\n 55,\n 45,\n 52,\n 56,\n 53,\n 52,\n 45,\n 57,\n 98,\n 55,\n 99,\n 45,\n 97,\n 49,\n 101,\n 97,\n 97,\n 99,\n 49,\n 97,\n 49,\n 97,\n 56,\n 50,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60428 req_id = 9bdfd14e-69a9-44ea-99ea-339b5e420141
20402024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (store): query key key = b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host
20412024-11-15T20:11:30.832ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60428 query = LowerQuery { name: LowerName(Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal.")), original: Query { name: Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9bdfd14e-69a9-44ea-99ea-339b5e420141
20422024-11-15T20:11:30.832ZINFOtest_mgs_metrics (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:36837] } background_task = dns_servers_internal dns_group = internal
20432024-11-15T20:11:30.833ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 64.576778ms iteration = 2 status = {"addresses":["[::1]:36837"]}
20442024-11-15T20:11:30.833ZINFOtest_mgs_metrics (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:40887] } background_task = dns_servers_external dns_group = external
20452024-11-15T20:11:30.833ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 63.425286ms iteration = 2 status = {"addresses":["[::1]:40887"]}
20462024-11-15T20:11:30.833ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
20472024-11-15T20:11:30.835ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config
20482024-11-15T20:11:30.835ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 122.777\u{b5}s iteration = 2 status = {"error":"no config"}
20492024-11-15T20:11:30.835ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
20502024-11-15T20:11:30.835ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config
20512024-11-15T20:11:30.835ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 17.301\u{b5}s iteration = 2 status = {"error":"no config"}
20522024-11-15T20:11:30.839ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
20532024-11-15T20:11:30.840ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
20542024-11-15T20:11:30.841ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20552024-11-15T20:11:30.848ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20562024-11-15T20:11:30.848ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20572024-11-15T20:11:30.848ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20582024-11-15T20:11:30.849ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20592024-11-15T20:11:30.849ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
20602024-11-15T20:11:30.849ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20612024-11-15T20:11:30.849ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20622024-11-15T20:11:30.850ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20632024-11-15T20:11:30.850ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20642024-11-15T20:11:30.851ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20652024-11-15T20:11:30.859ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
20662024-11-15T20:11:30.859ZINFOtest_mgs_metrics (nexus): Rack initialized
20672024-11-15T20:11:30.860ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20682024-11-15T20:11:30.860ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20692024-11-15T20:11:30.860ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20702024-11-15T20:11:30.874ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20712024-11-15T20:11:30.891ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20722024-11-15T20:11:30.891ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
20732024-11-15T20:11:30.891ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20742024-11-15T20:11:30.892ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20752024-11-15T20:11:30.896ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c8 f5 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57680 req_id = 3d2d9762-7b15-4b22-9517-e114e62e217f
20762024-11-15T20:11:30.896ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51445,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57680 req_id = 3d2d9762-7b15-4b22-9517-e114e62e217f
20772024-11-15T20:11:30.897ZDEBGtest_mgs_metrics (store): query key key = _mgs._tcp
20782024-11-15T20:11:30.897ZDEBGtest_mgs_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
20792024-11-15T20:11:30.898ZDEBGtest_mgs_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
20802024-11-15T20:11:30.898ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57680 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 45617, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 35733, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3d2d9762-7b15-4b22-9517-e114e62e217f
20812024-11-15T20:11:30.898ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20822024-11-15T20:11:30.906ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
20832024-11-15T20:11:30.906ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20842024-11-15T20:11:30.906ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20852024-11-15T20:11:30.907ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20862024-11-15T20:11:30.908ZDEBGtest_mgs_metrics (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
20872024-11-15T20:11:30.908ZDEBGtest_mgs_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7c184eef-9381-4028-a65c-7afadb2ad118 dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
20882024-11-15T20:11:30.908ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20892024-11-15T20:11:30.908ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20902024-11-15T20:11:30.908ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20912024-11-15T20:11:30.908ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20922024-11-15T20:11:30.909ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20932024-11-15T20:11:30.910ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 45617, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 35733, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2040, tv_nsec: 708195488 } })
20942024-11-15T20:11:30.911ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
20952024-11-15T20:11:30.911ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
20962024-11-15T20:11:30.911ZDEBGtest_mgs_metrics (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
20972024-11-15T20:11:30.911ZDEBGtest_mgs_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = da4dceac-026d-41d5-898a-d47999752a0c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
20982024-11-15T20:11:30.911ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
20992024-11-15T20:11:30.911ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21002024-11-15T20:11:30.912ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
21012024-11-15T20:11:30.912ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21022024-11-15T20:11:30.912ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21032024-11-15T20:11:30.912ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21042024-11-15T20:11:30.912ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21052024-11-15T20:11:30.914ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21062024-11-15T20:11:30.922ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 43 c5 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e C...........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56560 req_id = 46a8c2c1-fe19-4c73-bc90-7bea9621b22a
21072024-11-15T20:11:30.922ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17349,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56560 req_id = 46a8c2c1-fe19-4c73-bc90-7bea9621b22a
21082024-11-15T20:11:30.934ZDEBGtest_mgs_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
21092024-11-15T20:11:30.934ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56560 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 46a8c2c1-fe19-4c73-bc90-7bea9621b22a
21102024-11-15T20:11:30.935ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ca ef 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63844 req_id = ac8ba4e9-4a35-4e18-885b-0878b3df7d31
21112024-11-15T20:11:30.936ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51951,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63844 req_id = ac8ba4e9-4a35-4e18-885b-0878b3df7d31
21122024-11-15T20:11:30.936ZDEBGtest_mgs_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
21132024-11-15T20:11:30.936ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63844 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ac8ba4e9-4a35-4e18-885b-0878b3df7d31
21142024-11-15T20:11:30.936ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21152024-11-15T20:11:30.968ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21162024-11-15T20:11:30.968ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
21172024-11-15T20:11:30.969ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21182024-11-15T20:11:30.969ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21192024-11-15T20:11:30.970ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21202024-11-15T20:11:30.983ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21212024-11-15T20:11:30.988ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
21222024-11-15T20:11:30.988ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21232024-11-15T20:11:30.989ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21242024-11-15T20:11:31.148ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
21252024-11-15T20:11:31.162ZDEBGtest_mgs_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7c184eef-9381-4028-a65c-7afadb2ad118 dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
21262024-11-15T20:11:31.163ZDEBGtest_mgs_metrics (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
21272024-11-15T20:11:31.163ZINFOtest_mgs_metrics (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external generation = 2
21282024-11-15T20:11:31.163ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 394.659225ms iteration = 2 status = {"generation":2}
21292024-11-15T20:11:31.163ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
21302024-11-15T20:11:31.234ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:40887 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:40887] } uri = http://[::1]:40887/config
21312024-11-15T20:11:31.235ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
21322024-11-15T20:11:31.235ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21332024-11-15T20:11:31.235ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
21342024-11-15T20:11:31.235ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21352024-11-15T20:11:31.235ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21362024-11-15T20:11:31.238ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21372024-11-15T20:11:31.238ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21382024-11-15T20:11:31.238ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21392024-11-15T20:11:31.238ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:40887 remote_addr = [::1]:58982
21402024-11-15T20:11:31.238ZDEBGtest_mgs_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = da4dceac-026d-41d5-898a-d47999752a0c dns_zone_name = control-plane.oxide.internal found_so_far = 18 version = 1
21412024-11-15T20:11:31.239ZDEBGtest_mgs_metrics (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
21422024-11-15T20:11:31.239ZINFOtest_mgs_metrics (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal generation = 1
21432024-11-15T20:11:31.239ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 470.969565ms iteration = 2 status = {"generation":1}
21442024-11-15T20:11:31.239ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
21452024-11-15T20:11:31.280ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:36837 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:36837] } uri = http://[::1]:36837/config
21462024-11-15T20:11:31.282ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21472024-11-15T20:11:31.288ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21482024-11-15T20:11:31.289ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
21492024-11-15T20:11:31.289ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21502024-11-15T20:11:31.289ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21512024-11-15T20:11:31.290ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:40887 method = PUT remote_addr = [::1]:58982 req_id = ebb30169-ffc7-4636-b660-f67945db686f uri = /config
21522024-11-15T20:11:31.290ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:36837 remote_addr = [::1]:44306
21532024-11-15T20:11:31.290ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: b5 d0 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:65271 req_id = 1154033f-451a-45f5-90ee-d68c55ee757c
21542024-11-15T20:11:31.290ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46544,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65271 req_id = 1154033f-451a-45f5-90ee-d68c55ee757c
21552024-11-15T20:11:31.291ZDEBGtest_mgs_metrics (store): query key key = _clickhouse-admin-keeper._tcp
21562024-11-15T20:11:31.291ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 46544, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65271 req_id = 1154033f-451a-45f5-90ee-d68c55ee757c
21572024-11-15T20:11:31.297ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 2 req_id = ebb30169-ffc7-4636-b660-f67945db686f
21582024-11-15T20:11:31.297ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
21592024-11-15T20:11:31.297ZDEBGtest_mgs_metrics (store): creating tree tree_name = generation_2_zone_oxide-dev.test
21602024-11-15T20:11:31.298ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21612024-11-15T20:11:31.298ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21622024-11-15T20:11:31.298ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21632024-11-15T20:11:31.298ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21642024-11-15T20:11:31.299ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21652024-11-15T20:11:31.299ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:36837 method = PUT remote_addr = [::1]:44306 req_id = dbd76271-baed-4160-a1cd-37081c4fc54d uri = /config
21662024-11-15T20:11:31.299ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection background_task = inventory_collection
21672024-11-15T20:11:31.299ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:45617
21682024-11-15T20:11:31.299ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/ignition
21692024-11-15T20:11:31.300ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 1 req_id = dbd76271-baed-4160-a1cd-37081c4fc54d
21702024-11-15T20:11:31.300ZINFOtest_mgs_metrics (store): updated generation new_generation = 1 req_id = dbd76271-baed-4160-a1cd-37081c4fc54d
21712024-11-15T20:11:31.300ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 1509 local_addr = [::1]:36837 method = PUT remote_addr = [::1]:44306 req_id = dbd76271-baed-4160-a1cd-37081c4fc54d response_code = 204 uri = /config
21722024-11-15T20:11:31.300ZDEBGtest_mgs_metrics (store): updating current config
21732024-11-15T20:11:31.301ZDEBGtest_mgs_metrics (store): flushing default tree
21742024-11-15T20:11:31.304ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:36837 generation = 1 result = Ok(Response { url: "http://[::1]:36837/config", status: 204, headers: {"x-request-id": "dbd76271-baed-4160-a1cd-37081c4fc54d", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} }) servers = DnsServersList { addresses: [[::1]:36837] }
21752024-11-15T20:11:31.304ZINFOtest_mgs_metrics (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:36837 generation = 1 servers = DnsServersList { addresses: [[::1]:36837] }
21762024-11-15T20:11:31.305ZINFOtest_mgs_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:36837] }
21772024-11-15T20:11:31.305ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 66.712795ms iteration = 3 status = {"generation":1,"server_results":{"[::1]:36837":{"Ok":null}}}
21782024-11-15T20:11:31.305ZINFOtest_mgs_metrics (store): pruning trees for generations older than 2 keep = 3
21792024-11-15T20:11:31.305ZINFOtest_mgs_metrics (store): updated generation new_generation = 2 req_id = ebb30169-ffc7-4636-b660-f67945db686f
21802024-11-15T20:11:31.305ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 15485 local_addr = [::1]:40887 method = PUT remote_addr = [::1]:58982 req_id = ebb30169-ffc7-4636-b660-f67945db686f response_code = 204 uri = /config
21812024-11-15T20:11:31.307ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:40887 generation = 2 result = Ok(Response { url: "http://[::1]:40887/config", status: 204, headers: {"x-request-id": "ebb30169-ffc7-4636-b660-f67945db686f", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} }) servers = DnsServersList { addresses: [[::1]:40887] }
21822024-11-15T20:11:31.307ZINFOtest_mgs_metrics (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:40887 generation = 2 servers = DnsServersList { addresses: [[::1]:40887] }
21832024-11-15T20:11:31.308ZINFOtest_mgs_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:40887] }
21842024-11-15T20:11:31.308ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 144.899648ms iteration = 3 status = {"generation":2,"server_results":{"[::1]:40887":{"Ok":null}}}
21852024-11-15T20:11:31.309ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21862024-11-15T20:11:31.309ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21872024-11-15T20:11:31.309ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21882024-11-15T20:11:31.309ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21892024-11-15T20:11:31.335ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7de78db-94b8-43ea-bab2-6a38ef4a4215", "content-length": "835", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
21902024-11-15T20:11:31.336ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0
21912024-11-15T20:11:31.336ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21922024-11-15T20:11:31.336ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21932024-11-15T20:11:31.336ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
21942024-11-15T20:11:31.337ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21952024-11-15T20:11:31.344ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21962024-11-15T20:11:31.350ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21972024-11-15T20:11:31.350ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21982024-11-15T20:11:31.351ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21992024-11-15T20:11:31.351ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22002024-11-15T20:11:31.351ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22012024-11-15T20:11:31.351ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22022024-11-15T20:11:31.351ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22032024-11-15T20:11:31.352ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de3d4b40-13b9-4880-b5df-a21ead92e10e", "content-length": "734", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22042024-11-15T20:11:31.352ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/sp/caboose?firmware_slot=0
22052024-11-15T20:11:31.355ZINFOtest_mgs_metrics (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-15T20:11:23.368518Z, time_modified: 2024-11-15T20:11:23.368518Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: d6d76869-d8e8-42dd-92fb-71e6ce54721e, db_silo: Silo { identity: SiloIdentity { id: d6d76869-d8e8-42dd-92fb-71e6ce54721e, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-11-15T20:11:30.089298Z, time_modified: 2024-11-15T20:11:30.089298Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: d6d76869-d8e8-42dd-92fb-71e6ce54721e, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d6d76869-d8e8-42dd-92fb-71e6ce54721e, db_silo: Silo { identity: SiloIdentity { id: d6d76869-d8e8-42dd-92fb-71e6ce54721e, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-11-15T20:11:30.089298Z, time_modified: 2024-11-15T20:11:30.089298Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
22062024-11-15T20:11:31.355ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 584.540241ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"d6d76869-d8e8-42dd-92fb-71e6ce54721e","tls_certs":[]}},"default_endpoint":{"silo_id":"d6d76869-d8e8-42dd-92fb-71e6ce54721e","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo d6d76869-d8e8-42dd-92fb-71e6ce54721e with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
22072024-11-15T20:11:31.355ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22082024-11-15T20:11:31.355ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22092024-11-15T20:11:31.355ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22102024-11-15T20:11:31.356ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22112024-11-15T20:11:31.357ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3deba0b6-ee25-4205-98bd-5c496020641f", "content-length": "111", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22122024-11-15T20:11:31.357ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/sp/caboose?firmware_slot=1
22132024-11-15T20:11:31.359ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22142024-11-15T20:11:31.359ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22152024-11-15T20:11:31.359ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22162024-11-15T20:11:31.359ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22172024-11-15T20:11:31.362ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cc39c7a-4705-47f2-a5a7-3cf8fbd7df05", "content-length": "111", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22182024-11-15T20:11:31.362ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/rot/caboose?firmware_slot=0
22192024-11-15T20:11:31.362ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22202024-11-15T20:11:31.371ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
22212024-11-15T20:11:31.371ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22222024-11-15T20:11:31.371ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22232024-11-15T20:11:31.372ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22242024-11-15T20:11:31.375ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22252024-11-15T20:11:31.375ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22262024-11-15T20:11:31.375ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22272024-11-15T20:11:31.376ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0794e588-648c-4b5d-88d5-2729ca07174e", "content-length": "167", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22282024-11-15T20:11:31.377ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/rot/caboose?firmware_slot=1
22292024-11-15T20:11:31.378ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14f3b15d-4134-4e6b-8b0b-d4b0aec8ef4f", "content-length": "167", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22302024-11-15T20:11:31.379ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/stage0/caboose?firmware_slot=0
22312024-11-15T20:11:31.379ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22322024-11-15T20:11:31.379ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22332024-11-15T20:11:31.380ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22342024-11-15T20:11:31.382ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22352024-11-15T20:11:31.382ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22362024-11-15T20:11:31.382ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22372024-11-15T20:11:31.382ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd3fbbfb-3922-4eef-9d6b-a01b7eb6bd06", "content-length": "175", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22382024-11-15T20:11:31.383ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/stage0/caboose?firmware_slot=1
22392024-11-15T20:11:31.384ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
22402024-11-15T20:11:31.384ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22412024-11-15T20:11:31.384ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22422024-11-15T20:11:31.384ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22432024-11-15T20:11:31.385ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22442024-11-15T20:11:31.386ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22452024-11-15T20:11:31.387ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22462024-11-15T20:11:31.387ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22472024-11-15T20:11:31.387ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22482024-11-15T20:11:31.388ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "deac498c-6d7a-4f73-99ea-6d8221d89ed1", "content-length": "175", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22492024-11-15T20:11:31.389ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/rot/cmpa
22502024-11-15T20:11:31.389ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22512024-11-15T20:11:31.389ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22522024-11-15T20:11:31.389ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22532024-11-15T20:11:31.389ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22542024-11-15T20:11:31.391ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "467de3a8-7d68-4a7e-bece-6b1d86a11cfe", "content-length": "702", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22552024-11-15T20:11:31.391ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/rot/cfpa
22562024-11-15T20:11:31.392ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26b30c60-52dc-44a3-b43f-925a4fe2be34", "content-length": "727", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22572024-11-15T20:11:31.392ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/rot/cfpa
22582024-11-15T20:11:31.393ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22592024-11-15T20:11:31.402ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
22602024-11-15T20:11:31.402ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
22612024-11-15T20:11:31.402ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22622024-11-15T20:11:31.402ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22632024-11-15T20:11:31.403ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22642024-11-15T20:11:31.407ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6add191b-d3e8-4841-81b9-657715ef39f1", "content-length": "729", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22652024-11-15T20:11:31.407ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/0/component/rot/cfpa
22662024-11-15T20:11:31.407ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c90aa42-2b8f-45ff-9e12-c8ae991fe2d4", "content-length": "728", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22672024-11-15T20:11:31.407ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1
22682024-11-15T20:11:31.408ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22692024-11-15T20:11:31.408ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22702024-11-15T20:11:31.408ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22712024-11-15T20:11:31.408ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e49dc7f5-dc1a-41b6-b052-2ce7868ad876", "content-length": "734", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22722024-11-15T20:11:31.408ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/sp/caboose?firmware_slot=0
22732024-11-15T20:11:31.411ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b353bab5-65c4-4070-b878-71d8523359f4", "content-length": "111", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22742024-11-15T20:11:31.411ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/sp/caboose?firmware_slot=1
22752024-11-15T20:11:31.412ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22762024-11-15T20:11:31.412ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22772024-11-15T20:11:31.413ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22782024-11-15T20:11:31.415ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22792024-11-15T20:11:31.415ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22802024-11-15T20:11:31.415ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22812024-11-15T20:11:31.416ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa96ef9e-48ef-483a-9e81-2d236fa8ff02", "content-length": "111", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22822024-11-15T20:11:31.416ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/rot/caboose?firmware_slot=0
22832024-11-15T20:11:31.418ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22842024-11-15T20:11:31.418ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22852024-11-15T20:11:31.418ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22862024-11-15T20:11:31.418ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22872024-11-15T20:11:31.419ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22882024-11-15T20:11:31.421ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22892024-11-15T20:11:31.421ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22902024-11-15T20:11:31.421ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22912024-11-15T20:11:31.422ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22922024-11-15T20:11:31.423ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "043a50ec-c8c4-4ee8-90ee-f2e81d42c5dd", "content-length": "167", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
22932024-11-15T20:11:31.423ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/rot/caboose?firmware_slot=1
22942024-11-15T20:11:31.425ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22952024-11-15T20:11:31.425ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22962024-11-15T20:11:31.425ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22972024-11-15T20:11:31.425ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22982024-11-15T20:11:31.426ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22992024-11-15T20:11:31.433ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23002024-11-15T20:11:31.433ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
23012024-11-15T20:11:31.433ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
23022024-11-15T20:11:31.433ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23032024-11-15T20:11:31.433ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23042024-11-15T20:11:31.433ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23052024-11-15T20:11:31.434ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23062024-11-15T20:11:31.439ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db2b815a-524c-40f5-b5b7-aa0707d57b09", "content-length": "167", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23072024-11-15T20:11:31.439ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/stage0/caboose?firmware_slot=0
23082024-11-15T20:11:31.439ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23092024-11-15T20:11:31.439ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23102024-11-15T20:11:31.439ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23112024-11-15T20:11:31.439ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23122024-11-15T20:11:31.441ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23132024-11-15T20:11:31.441ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23142024-11-15T20:11:31.442ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23152024-11-15T20:11:31.442ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23162024-11-15T20:11:31.442ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1def5bc-6b9b-4ed5-9cce-d247680e3a59", "content-length": "175", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23172024-11-15T20:11:31.442ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/stage0/caboose?firmware_slot=1
23182024-11-15T20:11:31.442ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23192024-11-15T20:11:31.453ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23202024-11-15T20:11:31.454ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23212024-11-15T20:11:31.454ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23222024-11-15T20:11:31.454ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23232024-11-15T20:11:31.459ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23242024-11-15T20:11:31.459ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23252024-11-15T20:11:31.459ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23262024-11-15T20:11:31.460ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6499ff7a-3cc2-4158-b907-c5026587736a", "content-length": "175", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23272024-11-15T20:11:31.462ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/rot/cmpa
23282024-11-15T20:11:31.462ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23292024-11-15T20:11:31.462ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23302024-11-15T20:11:31.462ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23312024-11-15T20:11:31.463ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "652fcc28-fde1-4107-a386-6a1b30584419", "content-length": "702", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23322024-11-15T20:11:31.463ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/rot/cfpa
23332024-11-15T20:11:31.463ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23342024-11-15T20:11:31.463ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23352024-11-15T20:11:31.464ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23362024-11-15T20:11:31.465ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bee8545-0cdc-4b73-b5dc-866a5cd2aa6f", "content-length": "727", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23372024-11-15T20:11:31.465ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/rot/cfpa
23382024-11-15T20:11:31.466ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "84d69944-f4f8-4407-9d35-8b5c4ac2f69e", "content-length": "729", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23392024-11-15T20:11:31.466ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/switch/1/component/rot/cfpa
23402024-11-15T20:11:31.467ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23412024-11-15T20:11:31.467ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23422024-11-15T20:11:31.467ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23432024-11-15T20:11:31.470ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a59f38c-7d7a-4c84-9bc9-9deb6edf6ff7", "content-length": "728", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23442024-11-15T20:11:31.470ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0
23452024-11-15T20:11:31.471ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
23462024-11-15T20:11:31.471ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23472024-11-15T20:11:31.471ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23482024-11-15T20:11:31.471ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23492024-11-15T20:11:31.472ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23502024-11-15T20:11:31.473ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a98db01-de7f-4781-b640-3d56207d4ddf", "content-length": "723", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23512024-11-15T20:11:31.474ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/sp/caboose?firmware_slot=0
23522024-11-15T20:11:31.474ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23532024-11-15T20:11:31.474ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23542024-11-15T20:11:31.474ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23552024-11-15T20:11:31.474ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23562024-11-15T20:11:31.476ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "33f66be2-16bb-4e01-8874-457ef15f1f23", "content-length": "109", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23572024-11-15T20:11:31.476ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/sp/caboose?firmware_slot=1
23582024-11-15T20:11:31.477ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23592024-11-15T20:11:31.477ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23602024-11-15T20:11:31.477ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23612024-11-15T20:11:31.477ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23622024-11-15T20:11:31.481ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23632024-11-15T20:11:31.490ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23642024-11-15T20:11:31.492ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
23652024-11-15T20:11:31.492ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
23662024-11-15T20:11:31.492ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23672024-11-15T20:11:31.493ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23682024-11-15T20:11:31.493ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23692024-11-15T20:11:31.493ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23702024-11-15T20:11:31.493ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "abba122a-edd2-45b0-94f5-882c996d5aea", "content-length": "109", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23712024-11-15T20:11:31.493ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/rot/caboose?firmware_slot=0
23722024-11-15T20:11:31.494ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38fa2536-cc0f-49dd-b573-ac060902c316", "content-length": "169", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23732024-11-15T20:11:31.494ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/rot/caboose?firmware_slot=1
23742024-11-15T20:11:31.497ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e803388-24b3-4ebb-a038-122bbd9c67f5", "content-length": "169", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23752024-11-15T20:11:31.504ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/stage0/caboose?firmware_slot=0
23762024-11-15T20:11:31.504ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23772024-11-15T20:11:31.504ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23782024-11-15T20:11:31.504ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23792024-11-15T20:11:31.504ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23802024-11-15T20:11:31.504ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23812024-11-15T20:11:31.504ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23822024-11-15T20:11:31.504ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23832024-11-15T20:11:31.504ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23842024-11-15T20:11:31.504ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a65a2c7-873f-48cf-88d1-0f895f4c5934", "content-length": "178", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23852024-11-15T20:11:31.504ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/stage0/caboose?firmware_slot=1
23862024-11-15T20:11:31.506ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23872024-11-15T20:11:31.516ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23882024-11-15T20:11:31.516ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23892024-11-15T20:11:31.516ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23902024-11-15T20:11:31.517ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23912024-11-15T20:11:31.519ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3a46fa4-ab75-4ba4-af51-4f5632eb170d", "content-length": "178", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23922024-11-15T20:11:31.520ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/rot/cmpa
23932024-11-15T20:11:31.521ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bf7ce12-c339-4bf0-bd4d-8c912541a68f", "content-length": "702", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23942024-11-15T20:11:31.521ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/rot/cfpa
23952024-11-15T20:11:31.523ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec88fdb7-25f4-4aca-a0b1-4611c4f0ff49", "content-length": "727", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23962024-11-15T20:11:31.523ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/rot/cfpa
23972024-11-15T20:11:31.524ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8542dab-39df-4735-9dba-5d66079ba3eb", "content-length": "729", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
23982024-11-15T20:11:31.524ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/0/component/rot/cfpa
23992024-11-15T20:11:31.526ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "facc7b45-1df1-4d60-9f0f-0eb33e1611fa", "content-length": "728", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24002024-11-15T20:11:31.526ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1
24012024-11-15T20:11:31.528ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce37d29f-e3b4-435e-abd9-b93373ecc8cd", "content-length": "723", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24022024-11-15T20:11:31.528ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/sp/caboose?firmware_slot=0
24032024-11-15T20:11:31.529ZDEBGtest_mgs_metrics (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 9be58fc7-4cf7-4b1e-94eb-817a0b5a6927, kind: Service { id: c74391b6-2721-445b-b829-94c88c82a5d0 }, name: Name("nexus-c74391b6-2721-445b-b829-94c88c82a5d0"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 7c44a9cf-0a90-4b3d-a2cd-737ce92945f4, kind: Service { id: b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82 }, name: Name("external-dns-b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
24042024-11-15T20:11:31.530ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
24052024-11-15T20:11:31.530ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24062024-11-15T20:11:31.530ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24072024-11-15T20:11:31.531ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24082024-11-15T20:11:31.531ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24092024-11-15T20:11:31.533ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2a7ec3a-9c00-48e9-b94d-ba282412702a", "content-length": "109", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24102024-11-15T20:11:31.534ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/sp/caboose?firmware_slot=1
24112024-11-15T20:11:31.534ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24122024-11-15T20:11:31.541ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
24132024-11-15T20:11:31.541ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24142024-11-15T20:11:31.541ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24152024-11-15T20:11:31.542ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24162024-11-15T20:11:31.546ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6216470-6af5-4df6-bf05-9717e77bc2de", "content-length": "109", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24172024-11-15T20:11:31.547ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/rot/caboose?firmware_slot=0
24182024-11-15T20:11:31.547ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcf25752-cdcc-4c83-82dc-3c0d1dd61034", "content-length": "169", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24192024-11-15T20:11:31.547ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/rot/caboose?firmware_slot=1
24202024-11-15T20:11:31.549ZTRACtest_mgs_metrics (InternalApi): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-11-15T19:50:02.602910Z, time_modified: 2024-11-15T20:11:30.279470Z, allowed_ips: None }
24212024-11-15T20:11:31.549ZDEBGtest_mgs_metrics (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
24222024-11-15T20:11:31.549ZDEBGtest_mgs_metrics (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 7c44a9cf-0a90-4b3d-a2cd-737ce92945f4, kind: Service { id: b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82 }, name: Name("external-dns-b7c3dc55-c0a7-4854-9b7c-a1eaac1a1a82"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 9be58fc7-4cf7-4b1e-94eb-817a0b5a6927, kind: Service { id: c74391b6-2721-445b-b829-94c88c82a5d0 }, name: Name("nexus-c74391b6-2721-445b-b829-94c88c82a5d0"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
24232024-11-15T20:11:31.549ZDEBGtest_mgs_metrics (InternalApi): resolved 2 rules for sleds
24242024-11-15T20:11:31.552ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4fb87d6-c64f-41b0-b90c-e98fe6dab042", "content-length": "169", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24252024-11-15T20:11:31.552ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/stage0/caboose?firmware_slot=0
24262024-11-15T20:11:31.554ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb57da69-094c-46e3-aa95-6dafd36e29be", "content-length": "178", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24272024-11-15T20:11:31.554ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/stage0/caboose?firmware_slot=1
24282024-11-15T20:11:31.557ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d14dd4a-49fe-4f2c-8d1f-0282d49caaa1", "content-length": "178", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24292024-11-15T20:11:31.557ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/rot/cmpa
24302024-11-15T20:11:31.558ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "15a72afc-3a8b-427b-9aae-902a3f277324", "content-length": "702", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24312024-11-15T20:11:31.559ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/rot/cfpa
24322024-11-15T20:11:31.560ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "08fe59d1-dd38-4dca-8720-5f9d30d04e4b", "content-length": "727", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24332024-11-15T20:11:31.560ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/rot/cfpa
24342024-11-15T20:11:31.562ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1465a8b5-7372-416b-8e96-426353e94988", "content-length": "729", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24352024-11-15T20:11:31.562ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:45617 method = GET uri = http://[::1]:45617/sp/sled/1/component/rot/cfpa
24362024-11-15T20:11:31.563ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45617 result = Ok(Response { url: "http://[::1]:45617/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "98c5c953-fcb8-493e-b3b8-27bdff30ca7e", "content-length": "728", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24372024-11-15T20:11:31.564ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:35733
24382024-11-15T20:11:31.564ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:35733 method = GET uri = http://[::1]:35733/ignition
24392024-11-15T20:11:31.565ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:35733 result = Ok(Response { url: "http://[::1]:35733/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5bb9fbe-c4fb-4583-a811-e71b3e84f5f1", "content-length": "835", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24402024-11-15T20:11:31.566ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:35733 method = GET uri = http://[::1]:35733/sp/switch/0
24412024-11-15T20:11:31.566ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:35733 result = Ok(Response { url: "http://[::1]:35733/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78c66d62-63c3-4772-b7e0-262e2fffa378", "content-length": "734", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24422024-11-15T20:11:31.566ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:35733 method = GET uri = http://[::1]:35733/sp/switch/1
24432024-11-15T20:11:31.568ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:35733 result = Ok(Response { url: "http://[::1]:35733/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3572b7df-35d9-461f-acda-8e2ae3c4defa", "content-length": "734", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24442024-11-15T20:11:31.568ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:35733 method = GET uri = http://[::1]:35733/sp/sled/0
24452024-11-15T20:11:31.569ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:35733 result = Ok(Response { url: "http://[::1]:35733/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "29198ae4-d1ea-4924-9436-6f0e3e40cdda", "content-length": "723", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24462024-11-15T20:11:31.569ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:35733 method = GET uri = http://[::1]:35733/sp/sled/1
24472024-11-15T20:11:31.570ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:35733 result = Ok(Response { url: "http://[::1]:35733/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e836e209-4f06-4ca8-bd6b-e844f99b56a0", "content-length": "723", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24482024-11-15T20:11:31.571ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
24492024-11-15T20:11:31.571ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24502024-11-15T20:11:31.571ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24512024-11-15T20:11:31.571ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24522024-11-15T20:11:31.571ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24532024-11-15T20:11:31.574ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24542024-11-15T20:11:31.580ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24552024-11-15T20:11:31.580ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
24562024-11-15T20:11:31.580ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24572024-11-15T20:11:31.580ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24582024-11-15T20:11:31.580ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24592024-11-15T20:11:31.580ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24602024-11-15T20:11:31.588ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24612024-11-15T20:11:31.594ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24622024-11-15T20:11:31.595ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24632024-11-15T20:11:31.596ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24642024-11-15T20:11:31.596ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24652024-11-15T20:11:31.658ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39103
24662024-11-15T20:11:31.659ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = http://[::1]:39103 background_task = inventory_collection body = None method = GET uri = http://[::1]:39103/inventory
24672024-11-15T20:11:31.659ZDEBGtest_mgs_metrics (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2024-11-15T20:11:25.540956Z, time_modified: 2024-11-15T20:11:25.540956Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(45047), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
24682024-11-15T20:11:31.659ZDEBGtest_mgs_metrics (InternalApi): sending firewall rules to sled agents
24692024-11-15T20:11:31.659ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24702024-11-15T20:11:31.659ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24712024-11-15T20:11:31.660ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24722024-11-15T20:11:31.660ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:39103 remote_addr = [::1]:51756 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
24732024-11-15T20:11:31.661ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:39103 method = GET remote_addr = [::1]:51756 req_id = ab1d77b7-3f2d-4d78-a834-efc99bf4cf9d sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
24742024-11-15T20:11:31.662ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 762 local_addr = [::1]:39103 method = GET remote_addr = [::1]:51756 req_id = ab1d77b7-3f2d-4d78-a834-efc99bf4cf9d response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
24752024-11-15T20:11:31.663ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = http://[::1]:39103 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39103/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab1d77b7-3f2d-4d78-a834-efc99bf4cf9d", "content-length": "4199", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24762024-11-15T20:11:31.700ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:45047
24772024-11-15T20:11:31.700ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = http://[::1]:45047 background_task = inventory_collection body = None method = GET uri = http://[::1]:45047/inventory
24782024-11-15T20:11:31.701ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
24792024-11-15T20:11:31.702ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24802024-11-15T20:11:31.702ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24812024-11-15T20:11:31.702ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24822024-11-15T20:11:31.702ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24832024-11-15T20:11:31.702ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45047 remote_addr = [::1]:59860 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
24842024-11-15T20:11:31.703ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45047 method = GET remote_addr = [::1]:59860 req_id = 44ff3811-6c7d-45b7-a3f6-79284d9e20eb sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
24852024-11-15T20:11:31.709ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 5424 local_addr = [::1]:45047 method = GET remote_addr = [::1]:59860 req_id = 44ff3811-6c7d-45b7-a3f6-79284d9e20eb response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
24862024-11-15T20:11:31.710ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = http://[::1]:45047 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:45047/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "44ff3811-6c7d-45b7-a3f6-79284d9e20eb", "content-length": "6277", "date": "Fri, 15 Nov 2024 20:11:30 GMT"} })
24872024-11-15T20:11:31.711ZDEBGtest_mgs_metrics (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
24882024-11-15T20:11:31.711ZDEBGtest_mgs_metrics (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
24892024-11-15T20:11:31.712ZDEBGtest_mgs_metrics (BackgroundTasks): finished collection background_task = inventory_collection
24902024-11-15T20:11:31.714ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
24912024-11-15T20:11:31.715ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24922024-11-15T20:11:31.715ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
24932024-11-15T20:11:31.715ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24942024-11-15T20:11:31.715ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24952024-11-15T20:11:31.716ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24962024-11-15T20:11:31.721ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
24972024-11-15T20:11:31.765ZDEBGtest_mgs_metrics (InternalApi): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:45047/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
24982024-11-15T20:11:31.769ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:45047 remote_addr = [::1]:59862 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
24992024-11-15T20:11:31.776ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25002024-11-15T20:11:31.779ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
25012024-11-15T20:11:31.781ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
25022024-11-15T20:11:31.781ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
25032024-11-15T20:11:31.781ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
25042024-11-15T20:11:31.781ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59862 req_id = 606fc16d-143e-4862-8f54-378b3d6ab96f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
25052024-11-15T20:11:31.782ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 952 local_addr = [::1]:45047 method = PUT remote_addr = [::1]:59862 req_id = 606fc16d-143e-4862-8f54-378b3d6ab96f response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
25062024-11-15T20:11:31.786ZDEBGtest_mgs_metrics (InternalApi): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:45047/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "606fc16d-143e-4862-8f54-378b3d6ab96f", "date": "Fri, 15 Nov 2024 20:11:31 GMT"} })
25072024-11-15T20:11:31.787ZINFOtest_mgs_metrics (InternalApi): updated firewall rules on 1 sleds
25082024-11-15T20:11:31.788ZINFOtest_mgs_metrics (nexus): plumbed initial IP allowlist
25092024-11-15T20:11:31.824ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = / versions = all
25102024-11-15T20:11:31.824ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /access versions = all
25112024-11-15T20:11:31.824ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /assets/{path} versions = all
25122024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /device/auth versions = all
25132024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /device/confirm versions = all
25142024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /device/success versions = all
25152024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /device/token versions = all
25162024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /device/verify versions = all
25172024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /experimental/v1/probes versions = all
25182024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /experimental/v1/probes versions = all
25192024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /experimental/v1/probes/{probe} versions = all
25202024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /experimental/v1/probes/{probe} versions = all
25212024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /images versions = all
25222024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /login versions = all
25232024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /login/{silo_name}/local versions = all
25242024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
25252024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
25262024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
25272024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /lookup/{path} versions = all
25282024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /projects/{path} versions = all
25292024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /projects-new versions = all
25302024-11-15T20:11:31.825ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /settings/{path} versions = all
25312024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /system/{path} versions = all
25322024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /utilization versions = all
25332024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/certificates versions = all
25342024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/certificates versions = all
25352024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/certificates/{certificate} versions = all
25362024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/certificates/{certificate} versions = all
25372024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/disks versions = all
25382024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/disks versions = all
25392024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/disks/{disk} versions = all
25402024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/disks/{disk} versions = all
25412024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/disks/{disk}/bulk-write versions = all
25422024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
25432024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
25442024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/disks/{disk}/finalize versions = all
25452024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
25462024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/floating-ips versions = all
25472024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/floating-ips versions = all
25482024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
25492024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/floating-ips/{floating_ip} versions = all
25502024-11-15T20:11:31.826ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
25512024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
25522024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
25532024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/groups versions = all
25542024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/groups/{group_id} versions = all
25552024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/images versions = all
25562024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/images versions = all
25572024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/images/{image} versions = all
25582024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/images/{image} versions = all
25592024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/images/{image}/demote versions = all
25602024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/images/{image}/promote versions = all
25612024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/instances versions = all
25622024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/instances versions = all
25632024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/instances/{instance} versions = all
25642024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/instances/{instance} versions = all
25652024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/instances/{instance} versions = all
25662024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/instances/{instance}/disks versions = all
25672024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/instances/{instance}/disks/attach versions = all
25682024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/instances/{instance}/disks/detach versions = all
25692024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/instances/{instance}/external-ips versions = all
25702024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
25712024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
25722024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/instances/{instance}/reboot versions = all
25732024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/instances/{instance}/serial-console versions = all
25742024-11-15T20:11:31.827ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
25752024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
25762024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/instances/{instance}/start versions = all
25772024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/instances/{instance}/stop versions = all
25782024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/internet-gateway-ip-addresses versions = all
25792024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/internet-gateway-ip-addresses versions = all
25802024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
25812024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/internet-gateway-ip-pools versions = all
25822024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/internet-gateway-ip-pools versions = all
25832024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
25842024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/internet-gateways versions = all
25852024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/internet-gateways versions = all
25862024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
25872024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/internet-gateways/{gateway} versions = all
25882024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/ip-pools versions = all
25892024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/ip-pools/{pool} versions = all
25902024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/login/{silo_name}/local versions = all
25912024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/logout versions = all
25922024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/me versions = all
25932024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/me/groups versions = all
25942024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/me/ssh-keys versions = all
25952024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/me/ssh-keys versions = all
25962024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
25972024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
25982024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/metrics/{metric_name} versions = all
25992024-11-15T20:11:31.828ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/network-interfaces versions = all
26002024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/network-interfaces versions = all
26012024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/network-interfaces/{interface} versions = all
26022024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/network-interfaces/{interface} versions = all
26032024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/network-interfaces/{interface} versions = all
26042024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/ping versions = all
26052024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/policy versions = all
26062024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/policy versions = all
26072024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/projects versions = all
26082024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/projects versions = all
26092024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/projects/{project} versions = all
26102024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/projects/{project} versions = all
26112024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/projects/{project} versions = all
26122024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/projects/{project}/policy versions = all
26132024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/projects/{project}/policy versions = all
26142024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/snapshots versions = all
26152024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/snapshots versions = all
26162024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/snapshots/{snapshot} versions = all
26172024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/snapshots/{snapshot} versions = all
26182024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/disks versions = all
26192024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
26202024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/racks versions = all
26212024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
26222024-11-15T20:11:31.829ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/sleds versions = all
26232024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/hardware/sleds versions = all
26242024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
26252024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
26262024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
26272024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
26282024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
26292024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/switch-port versions = all
26302024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
26312024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
26322024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
26332024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/switches versions = all
26342024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
26352024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/identity-providers versions = all
26362024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/identity-providers/local/users versions = all
26372024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
26382024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
26392024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/identity-providers/saml versions = all
26402024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
26412024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/ip-pools versions = all
26422024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/ip-pools versions = all
26432024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
26442024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/ip-pools/{pool} versions = all
26452024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/ip-pools/{pool} versions = all
26462024-11-15T20:11:31.830ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
26472024-11-15T20:11:31.831ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
26482024-11-15T20:11:31.831ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
26492024-11-15T20:11:31.831ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
26502024-11-15T20:11:31.831ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
26512024-11-15T20:11:31.832ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
26522024-11-15T20:11:31.832ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
26532024-11-15T20:11:31.832ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
26542024-11-15T20:11:31.832ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/ip-pools-service versions = all
26552024-11-15T20:11:31.832ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/ip-pools-service/ranges versions = all
26562024-11-15T20:11:31.832ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
26572024-11-15T20:11:31.832ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
26582024-11-15T20:11:31.832ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/metrics/{metric_name} versions = all
26592024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/address-lot versions = all
26602024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/networking/address-lot versions = all
26612024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
26622024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
26632024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/allow-list versions = all
26642024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/networking/allow-list versions = all
26652024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/networking/bfd-disable versions = all
26662024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/networking/bfd-enable versions = all
26672024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/bfd-status versions = all
26682024-11-15T20:11:31.833ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/networking/bgp versions = all
26692024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/bgp versions = all
26702024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/networking/bgp versions = all
26712024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/bgp-announce-set versions = all
26722024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
26732024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
26742024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
26752024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/bgp-exported versions = all
26762024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/bgp-message-history versions = all
26772024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
26782024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/bgp-status versions = all
26792024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/loopback-address versions = all
26802024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/networking/loopback-address versions = all
26812024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
26822024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
26832024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/switch-port-settings versions = all
26842024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/networking/switch-port-settings versions = all
26852024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
26862024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/policy versions = all
26872024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/policy versions = all
26882024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/roles versions = all
26892024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/roles/{role_name} versions = all
26902024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/silo-quotas versions = all
26912024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/silos versions = all
26922024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/silos versions = all
26932024-11-15T20:11:31.834ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/system/silos/{silo} versions = all
26942024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/silos/{silo} versions = all
26952024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
26962024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/silos/{silo}/policy versions = all
26972024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/silos/{silo}/policy versions = all
26982024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/silos/{silo}/quotas versions = all
26992024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
27002024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/system/timeseries/query versions = all
27012024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/timeseries/schemas versions = all
27022024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/system/update/repository versions = all
27032024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/update/repository/{system_version} versions = all
27042024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/users versions = all
27052024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/users/{user_id} versions = all
27062024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/users-builtin versions = all
27072024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/users-builtin/{user} versions = all
27082024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/utilization/silos versions = all
27092024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/system/utilization/silos/{silo} versions = all
27102024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/users versions = all
27112024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/utilization versions = all
27122024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpc-firewall-rules versions = all
27132024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/vpc-firewall-rules versions = all
27142024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpc-router-routes versions = all
27152024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/vpc-router-routes versions = all
27162024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
27172024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpc-router-routes/{route} versions = all
27182024-11-15T20:11:31.835ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/vpc-router-routes/{route} versions = all
27192024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpc-routers versions = all
27202024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/vpc-routers versions = all
27212024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/vpc-routers/{router} versions = all
27222024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpc-routers/{router} versions = all
27232024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/vpc-routers/{router} versions = all
27242024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpc-subnets versions = all
27252024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/vpc-subnets versions = all
27262024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
27272024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpc-subnets/{subnet} versions = all
27282024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
27292024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
27302024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpcs versions = all
27312024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = POST path = /v1/vpcs versions = all
27322024-11-15T20:11:31.836ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = DELETE path = /v1/vpcs/{vpc} versions = all
27332024-11-15T20:11:31.840ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = GET path = /v1/vpcs/{vpc} versions = all
27342024-11-15T20:11:31.840ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:38265 method = PUT path = /v1/vpcs/{vpc} versions = all
27352024-11-15T20:11:31.840ZINFOtest_mgs_metrics (dropshot_external): listening local_addr = 127.0.0.1:38265
27362024-11-15T20:11:31.840ZDEBGtest_mgs_metrics (dropshot_external): successfully registered DTrace USDT probes local_addr = 127.0.0.1:38265
27372024-11-15T20:11:31.874ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = / versions = all
27382024-11-15T20:11:31.874ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /access versions = all
27392024-11-15T20:11:31.874ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /assets/{path} versions = all
27402024-11-15T20:11:31.874ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /device/auth versions = all
27412024-11-15T20:11:31.874ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /device/confirm versions = all
27422024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /device/success versions = all
27432024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /device/token versions = all
27442024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /device/verify versions = all
27452024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /experimental/v1/probes versions = all
27462024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /experimental/v1/probes versions = all
27472024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /experimental/v1/probes/{probe} versions = all
27482024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /experimental/v1/probes/{probe} versions = all
27492024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /images versions = all
27502024-11-15T20:11:31.875ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /login versions = all
27512024-11-15T20:11:31.877ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /login/{silo_name}/local versions = all
27522024-11-15T20:11:31.877ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
27532024-11-15T20:11:31.877ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
27542024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
27552024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /lookup/{path} versions = all
27562024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /projects/{path} versions = all
27572024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /projects-new versions = all
27582024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /settings/{path} versions = all
27592024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /system/{path} versions = all
27602024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /utilization versions = all
27612024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/certificates versions = all
27622024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/certificates versions = all
27632024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/certificates/{certificate} versions = all
27642024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/certificates/{certificate} versions = all
27652024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/disks versions = all
27662024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/disks versions = all
27672024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/disks/{disk} versions = all
27682024-11-15T20:11:31.878ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/disks/{disk} versions = all
27692024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/disks/{disk}/bulk-write versions = all
27702024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
27712024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
27722024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/disks/{disk}/finalize versions = all
27732024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
27742024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/floating-ips versions = all
27752024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/floating-ips versions = all
27762024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
27772024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/floating-ips/{floating_ip} versions = all
27782024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
27792024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
27802024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
27812024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/groups versions = all
27822024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/groups/{group_id} versions = all
27832024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/images versions = all
27842024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/images versions = all
27852024-11-15T20:11:31.879ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/images/{image} versions = all
27862024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/images/{image} versions = all
27872024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/images/{image}/demote versions = all
27882024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/images/{image}/promote versions = all
27892024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/instances versions = all
27902024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/instances versions = all
27912024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/instances/{instance} versions = all
27922024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/instances/{instance} versions = all
27932024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/instances/{instance} versions = all
27942024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/instances/{instance}/disks versions = all
27952024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/instances/{instance}/disks/attach versions = all
27962024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/instances/{instance}/disks/detach versions = all
27972024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/instances/{instance}/external-ips versions = all
27982024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27992024-11-15T20:11:31.880ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
28002024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/instances/{instance}/reboot versions = all
28012024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/instances/{instance}/serial-console versions = all
28022024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
28032024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
28042024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/instances/{instance}/start versions = all
28052024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/instances/{instance}/stop versions = all
28062024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/internet-gateway-ip-addresses versions = all
28072024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/internet-gateway-ip-addresses versions = all
28082024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
28092024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/internet-gateway-ip-pools versions = all
28102024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/internet-gateway-ip-pools versions = all
28112024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
28122024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/internet-gateways versions = all
28132024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/internet-gateways versions = all
28142024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
28152024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/internet-gateways/{gateway} versions = all
28162024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/ip-pools versions = all
28172024-11-15T20:11:31.881ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/ip-pools/{pool} versions = all
28182024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/login/{silo_name}/local versions = all
28192024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/logout versions = all
28202024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/me versions = all
28212024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/me/groups versions = all
28222024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/me/ssh-keys versions = all
28232024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/me/ssh-keys versions = all
28242024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
28252024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
28262024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/metrics/{metric_name} versions = all
28272024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/network-interfaces versions = all
28282024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/network-interfaces versions = all
28292024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/network-interfaces/{interface} versions = all
28302024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/network-interfaces/{interface} versions = all
28312024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/network-interfaces/{interface} versions = all
28322024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/ping versions = all
28332024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/policy versions = all
28342024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/policy versions = all
28352024-11-15T20:11:31.882ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/projects versions = all
28362024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/projects versions = all
28372024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/projects/{project} versions = all
28382024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/projects/{project} versions = all
28392024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/projects/{project} versions = all
28402024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/projects/{project}/policy versions = all
28412024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/projects/{project}/policy versions = all
28422024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/snapshots versions = all
28432024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/snapshots versions = all
28442024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/snapshots/{snapshot} versions = all
28452024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/snapshots/{snapshot} versions = all
28462024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/disks versions = all
28472024-11-15T20:11:31.883ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
28482024-11-15T20:11:31.886ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/racks versions = all
28492024-11-15T20:11:31.895ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
28502024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/sleds versions = all
28512024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/hardware/sleds versions = all
28522024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
28532024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
28542024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
28552024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
28562024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
28572024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/switch-port versions = all
28582024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
28592024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
28602024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
28612024-11-15T20:11:31.896ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/switches versions = all
28622024-11-15T20:11:31.897ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
28632024-11-15T20:11:31.898ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/identity-providers versions = all
28642024-11-15T20:11:31.898ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/identity-providers/local/users versions = all
28652024-11-15T20:11:31.898ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
28662024-11-15T20:11:31.898ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
28672024-11-15T20:11:31.898ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/identity-providers/saml versions = all
28682024-11-15T20:11:31.898ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
28692024-11-15T20:11:31.898ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/ip-pools versions = all
28702024-11-15T20:11:31.899ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/ip-pools versions = all
28712024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
28722024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/ip-pools/{pool} versions = all
28732024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/ip-pools/{pool} versions = all
28742024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
28752024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
28762024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
28772024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
28782024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
28792024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28802024-11-15T20:11:31.900ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28812024-11-15T20:11:31.901ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
28822024-11-15T20:11:31.901ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/ip-pools-service versions = all
28832024-11-15T20:11:31.901ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/ip-pools-service/ranges versions = all
28842024-11-15T20:11:31.903ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
28852024-11-15T20:11:31.903ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
28862024-11-15T20:11:31.903ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/metrics/{metric_name} versions = all
28872024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/address-lot versions = all
28882024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/networking/address-lot versions = all
28892024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
28902024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
28912024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/allow-list versions = all
28922024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/networking/allow-list versions = all
28932024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/networking/bfd-disable versions = all
28942024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/networking/bfd-enable versions = all
28952024-11-15T20:11:31.904ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/bfd-status versions = all
28962024-11-15T20:11:31.905ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/networking/bgp versions = all
28972024-11-15T20:11:31.905ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/bgp versions = all
28982024-11-15T20:11:31.905ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/networking/bgp versions = all
28992024-11-15T20:11:31.905ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/bgp-announce-set versions = all
29002024-11-15T20:11:31.905ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
29012024-11-15T20:11:31.905ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
29022024-11-15T20:11:31.905ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
29032024-11-15T20:11:31.906ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/bgp-exported versions = all
29042024-11-15T20:11:31.906ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/bgp-message-history versions = all
29052024-11-15T20:11:31.907ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
29062024-11-15T20:11:31.907ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/bgp-status versions = all
29072024-11-15T20:11:31.907ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/loopback-address versions = all
29082024-11-15T20:11:31.907ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/networking/loopback-address versions = all
29092024-11-15T20:11:31.907ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
29102024-11-15T20:11:31.907ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
29112024-11-15T20:11:31.907ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/switch-port-settings versions = all
29122024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/networking/switch-port-settings versions = all
29132024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
29142024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/policy versions = all
29152024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/policy versions = all
29162024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/roles versions = all
29172024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/roles/{role_name} versions = all
29182024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/silo-quotas versions = all
29192024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/silos versions = all
29202024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/silos versions = all
29212024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/system/silos/{silo} versions = all
29222024-11-15T20:11:31.908ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/silos/{silo} versions = all
29232024-11-15T20:11:31.909ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
29242024-11-15T20:11:31.909ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/silos/{silo}/policy versions = all
29252024-11-15T20:11:31.909ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/silos/{silo}/policy versions = all
29262024-11-15T20:11:31.909ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/silos/{silo}/quotas versions = all
29272024-11-15T20:11:31.909ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
29282024-11-15T20:11:31.909ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/system/timeseries/query versions = all
29292024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/timeseries/schemas versions = all
29302024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/system/update/repository versions = all
29312024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/update/repository/{system_version} versions = all
29322024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/users versions = all
29332024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/users/{user_id} versions = all
29342024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/users-builtin versions = all
29352024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/users-builtin/{user} versions = all
29362024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/utilization/silos versions = all
29372024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/system/utilization/silos/{silo} versions = all
29382024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/users versions = all
29392024-11-15T20:11:31.910ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/utilization versions = all
29402024-11-15T20:11:31.911ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpc-firewall-rules versions = all
29412024-11-15T20:11:31.911ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/vpc-firewall-rules versions = all
29422024-11-15T20:11:31.911ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpc-router-routes versions = all
29432024-11-15T20:11:31.911ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/vpc-router-routes versions = all
29442024-11-15T20:11:31.911ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
29452024-11-15T20:11:31.911ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpc-router-routes/{route} versions = all
29462024-11-15T20:11:31.911ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/vpc-router-routes/{route} versions = all
29472024-11-15T20:11:31.911ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpc-routers versions = all
29482024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/vpc-routers versions = all
29492024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/vpc-routers/{router} versions = all
29502024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpc-routers/{router} versions = all
29512024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/vpc-routers/{router} versions = all
29522024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpc-subnets versions = all
29532024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/vpc-subnets versions = all
29542024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
29552024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpc-subnets/{subnet} versions = all
29562024-11-15T20:11:31.912ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
29572024-11-15T20:11:31.913ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
29582024-11-15T20:11:31.913ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpcs versions = all
29592024-11-15T20:11:31.913ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = POST path = /v1/vpcs versions = all
29602024-11-15T20:11:31.913ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = DELETE path = /v1/vpcs/{vpc} versions = all
29612024-11-15T20:11:31.913ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = GET path = /v1/vpcs/{vpc} versions = all
29622024-11-15T20:11:31.913ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:44477 method = PUT path = /v1/vpcs/{vpc} versions = all
29632024-11-15T20:11:31.913ZINFOtest_mgs_metrics (dropshot_external_techport): listening local_addr = 127.0.0.1:44477
29642024-11-15T20:11:31.913ZDEBGtest_mgs_metrics (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = 127.0.0.1:44477
29652024-11-15T20:11:31.913ZDEBGtest_mgs_metrics (nexus-producer-server): registered DTrace probes
29662024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = 127.0.0.1:43967 method = GET path = /{producer_id} versions = all
29672024-11-15T20:11:31.914ZINFOtest_mgs_metrics (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = 127.0.0.1:43967
29682024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = 127.0.0.1:43967
29692024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (nexus-producer-server): Requested any available port, Dropshot server has been bound to 127.0.0.1:43967
29702024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (nexus-producer-server): Nexus IP provided explicitly, will use it addr = 127.0.0.1:37069
29712024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (nexus-producer-server): starting producer registration task
29722024-11-15T20:11:31.914ZINFOtest_mgs_metrics (nexus-producer-server): starting oximeter metric producer server address = 127.0.0.1:43967 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = c74391b6-2721-445b-b829-94c88c82a5d0
29732024-11-15T20:11:31.914ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29742024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29752024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29762024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (producer-registration-task): registering / renewing oximeter producer lease with Nexus
29772024-11-15T20:11:31.914ZDEBGtest_mgs_metrics (producer-registration-task): using nexus address for registration addr = 127.0.0.1:37069
29782024-11-15T20:11:31.923ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:37069/metrics/producers
29792024-11-15T20:11:31.924ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44682
29802024-11-15T20:11:31.924ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44682 req_id = 6c722958-3f35-42c3-8f15-da034cd9923c uri = /metrics/producers
29812024-11-15T20:11:31.925ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44682 req_id = 6c722958-3f35-42c3-8f15-da034cd9923c resource = Database uri = /metrics/producers
29822024-11-15T20:11:31.925ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44682 req_id = 6c722958-3f35-42c3-8f15-da034cd9923c roles = RoleSet { roles: {} } uri = /metrics/producers
29832024-11-15T20:11:31.926ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44682 req_id = 6c722958-3f35-42c3-8f15-da034cd9923c resource = Database result = Ok(()) uri = /metrics/producers
29842024-11-15T20:11:31.928ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
29852024-11-15T20:11:31.928ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29862024-11-15T20:11:31.928ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29872024-11-15T20:11:31.928ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29882024-11-15T20:11:31.928ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29892024-11-15T20:11:31.952ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29902024-11-15T20:11:31.956ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
29912024-11-15T20:11:31.956ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
29922024-11-15T20:11:31.956ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29932024-11-15T20:11:31.956ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29942024-11-15T20:11:31.957ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
29952024-11-15T20:11:31.957ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
29962024-11-15T20:11:31.982ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29972024-11-15T20:11:31.993ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
29982024-11-15T20:11:32.000ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
29992024-11-15T20:11:32.001ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
30002024-11-15T20:11:32.002ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
30012024-11-15T20:11:32.024ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 99826 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44682 req_id = 6c722958-3f35-42c3-8f15-da034cd9923c response_code = 503 uri = /metrics/producers
30022024-11-15T20:11:32.033ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:37069/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "6c722958-3f35-42c3-8f15-da034cd9923c", "content-length": "133", "date": "Fri, 15 Nov 2024 20:11:31 GMT"} })
30032024-11-15T20:11:32.035ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 298.415674ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"6c722958-3f35-42c3-8f15-da034cd9923c\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 15 Nov 2024 20:11:31 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"6c722958-3f35-42c3-8f15-da034cd9923c\\" }" file = oximeter/producer/src/lib.rs:420
30042024-11-15T20:11:32.118ZDEBGtest_mgs_metrics (init_with_steps): Running step start_oximeter
30052024-11-15T20:11:32.118ZDEBGtest_mgs_metrics: Starting Oximeter
30062024-11-15T20:11:32.118ZDEBGtest_mgs_metrics (oximeter): registered DTrace probes
30072024-11-15T20:11:32.118ZINFOtest_mgs_metrics (oximeter): starting oximeter server file = oximeter/collector/src/lib.rs:233
30082024-11-15T20:11:32.118ZDEBGtest_mgs_metrics (oximeter): creating ClickHouse client
30092024-11-15T20:11:32.118ZDEBGtest_mgs_metrics (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
30102024-11-15T20:11:32.118ZDEBGtest_mgs_metrics (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
30112024-11-15T20:11:32.118ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = SELECT MAX(value) AS max_version FROM oximeter.version;
30122024-11-15T20:11:32.450ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:37069/metrics/producers
30132024-11-15T20:11:32.451ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44684
30142024-11-15T20:11:32.453ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44684 req_id = 0dc06d26-8793-412e-a442-a0f52e177f7a uri = /metrics/producers
30152024-11-15T20:11:32.454ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44684 req_id = 0dc06d26-8793-412e-a442-a0f52e177f7a resource = Database uri = /metrics/producers
30162024-11-15T20:11:32.454ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44684 req_id = 0dc06d26-8793-412e-a442-a0f52e177f7a roles = RoleSet { roles: {} } uri = /metrics/producers
30172024-11-15T20:11:32.454ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44684 req_id = 0dc06d26-8793-412e-a442-a0f52e177f7a resource = Database result = Ok(()) uri = /metrics/producers
30182024-11-15T20:11:32.455ZWARNtest_mgs_metrics (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:942 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
30192024-11-15T20:11:32.455ZDEBGtest_mgs_metrics (oximeter-agent): oximeter database does not exist, creating collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
30202024-11-15T20:11:32.455ZINFOtest_mgs_metrics (clickhouse-client): reading db version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:851 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
30212024-11-15T20:11:32.455ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = SELECT MAX(value) AS max_version FROM oximeter.version;
30222024-11-15T20:11:32.457ZWARNtest_mgs_metrics (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:942 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
30232024-11-15T20:11:32.457ZINFOtest_mgs_metrics (clickhouse-client): read oximeter database version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:855 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 version = 0
30242024-11-15T20:11:32.458ZINFOtest_mgs_metrics (clickhouse-client): wiping and re-initializing oximeter schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:860 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
30252024-11-15T20:11:32.458ZDEBGtest_mgs_metrics (clickhouse-client): wiping ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
30262024-11-15T20:11:32.458ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = DROP DATABASE IF EXISTS oximeter
30272024-11-15T20:11:32.459ZDEBGtest_mgs_metrics (clickhouse-client): initializing ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
30282024-11-15T20:11:32.459ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = CREATE DATABASE IF NOT EXISTS oximeter
30292024-11-15T20:11:32.473ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\n/* The version table contains metadata about the `oximeter` database */\nCREATE TABLE IF NOT EXISTS oximeter.version\n(\n value UInt64,\n timestamp DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (value, timestamp)
30302024-11-15T20:11:32.511ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\n/* The measurement tables contain all individual samples from each timeseries.\n *\n * Each table stores a single datum type, and otherwise contains nearly the same\n * structure. The primary sorting key is on the timeseries name, key, and then\n * timestamp, so that all timeseries from the same schema are grouped, followed\n * by all samples from the same timeseries.\n *\n * This reflects that one usually looks up the _key_ in one or more field table,\n * and then uses that to index quickly into the measurements tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Bool)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30312024-11-15T20:11:32.525ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30322024-11-15T20:11:32.534ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 80421 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44684 req_id = 0dc06d26-8793-412e-a442-a0f52e177f7a response_code = 503 uri = /metrics/producers
30332024-11-15T20:11:32.534ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:37069/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "0dc06d26-8793-412e-a442-a0f52e177f7a", "content-length": "133", "date": "Fri, 15 Nov 2024 20:11:31 GMT"} })
30342024-11-15T20:11:32.535ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 631.372533ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"0dc06d26-8793-412e-a442-a0f52e177f7a\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 15 Nov 2024 20:11:31 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"0dc06d26-8793-412e-a442-a0f52e177f7a\\" }" file = oximeter/producer/src/lib.rs:420
30352024-11-15T20:11:32.541ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30362024-11-15T20:11:32.564ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30372024-11-15T20:11:32.580ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30382024-11-15T20:11:32.592ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30392024-11-15T20:11:32.609ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30402024-11-15T20:11:32.622ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30412024-11-15T20:11:32.635ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30422024-11-15T20:11:32.657ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30432024-11-15T20:11:32.686ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30442024-11-15T20:11:32.692ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 82 c8 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:64797 req_id = fc4154bf-f1d1-46c0-bdc9-40596ec7d94d
30452024-11-15T20:11:32.696ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33480,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64797 req_id = fc4154bf-f1d1-46c0-bdc9-40596ec7d94d
30462024-11-15T20:11:32.696ZDEBGtest_mgs_metrics (store): query key key = _crucible-pantry._tcp
30472024-11-15T20:11:32.696ZDEBGtest_mgs_metrics (store): query key key = c267c503-eaa6-44d1-b2b6-d6c4caa26255.host
30482024-11-15T20:11:32.696ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("c267c503-eaa6-44d1-b2b6-d6c4caa26255.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64797 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39023, target: Name("c267c503-eaa6-44d1-b2b6-d6c4caa26255.host.control-plane.oxide.internal") })) }] req_id = fc4154bf-f1d1-46c0-bdc9-40596ec7d94d
30492024-11-15T20:11:32.696ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: d9 1b 01 00 00 01 00 00 00 00 00 01 24 63 32 36 ............$c26\n0010: 37 63 35 30 33 2d 65 61 61 36 2d 34 34 64 31 2d 7c503-eaa6-44d1-\n0020: 62 32 62 36 2d 64 36 63 34 63 61 61 32 36 32 35 b2b6-d6c4caa2625\n0030: 35 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 5.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:59833 req_id = 9f911f58-22d2-4598-99e7-c1df3f41e06e
30502024-11-15T20:11:32.696ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55579,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("c267c503-eaa6-44d1-b2b6-d6c4caa26255.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("c267c503-eaa6-44d1-b2b6-d6c4caa26255.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 99,\n 50,\n 54,\n 55,\n 99,\n 53,\n 48,\n 51,\n 45,\n 101,\n 97,\n 97,\n 54,\n 45,\n 52,\n 52,\n 100,\n 49,\n 45,\n 98,\n 50,\n 98,\n 54,\n 45,\n 100,\n 54,\n 99,\n 52,\n 99,\n 97,\n 97,\n 50,\n 54,\n 50,\n 53,\n 53,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59833 req_id = 9f911f58-22d2-4598-99e7-c1df3f41e06e
30512024-11-15T20:11:32.696ZDEBGtest_mgs_metrics (store): query key key = c267c503-eaa6-44d1-b2b6-d6c4caa26255.host
30522024-11-15T20:11:32.696ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59833 query = LowerQuery { name: LowerName(Name("c267c503-eaa6-44d1-b2b6-d6c4caa26255.host.control-plane.oxide.internal.")), original: Query { name: Name("c267c503-eaa6-44d1-b2b6-d6c4caa26255.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("c267c503-eaa6-44d1-b2b6-d6c4caa26255.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9f911f58-22d2-4598-99e7-c1df3f41e06e
30532024-11-15T20:11:32.752ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(String)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30542024-11-15T20:11:32.754ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39023 remote_addr = [::1]:36486 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30552024-11-15T20:11:32.755ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39023 method = GET remote_addr = [::1]:36486 req_id = 8318f6a0-1a34-4c8a-850c-aabc369df15d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30562024-11-15T20:11:32.769ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 13526 local_addr = [::1]:39023 method = GET remote_addr = [::1]:36486 req_id = 8318f6a0-1a34-4c8a-850c-aabc369df15d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30572024-11-15T20:11:32.769ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bytes\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Right now we can't unambiguously record a nullable byte array.\n * Arrays cannot be nested in `Nullable()` types, and encoding the array as\n * a string isn't palatable for a few reasons.\n * See: https://github.com/oxidecomputer/omicron/issues/4551 for more\n * details.\n */\n datum Array(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30582024-11-15T20:11:33.122ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39023 remote_addr = [::1]:36488 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30592024-11-15T20:11:33.122ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39023 remote_addr = [::1]:36490 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30602024-11-15T20:11:33.122ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39023 remote_addr = [::1]:36492 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30612024-11-15T20:11:33.123ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39023 remote_addr = [::1]:36494 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30622024-11-15T20:11:33.123ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39023 remote_addr = [::1]:36496 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30632024-11-15T20:11:33.124ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39023 remote_addr = [::1]:36498 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30642024-11-15T20:11:33.125ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39023 remote_addr = [::1]:36500 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30652024-11-15T20:11:33.125ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39023 method = GET remote_addr = [::1]:36488 req_id = 5c6965ad-95ed-4f6b-9926-f2df10bbca27 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30662024-11-15T20:11:33.125ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39023 method = GET remote_addr = [::1]:36490 req_id = 1913ace3-df31-42e5-9edd-41d5621e368b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30672024-11-15T20:11:33.125ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39023 method = GET remote_addr = [::1]:36492 req_id = e4f149ec-0fc9-4baf-9182-bfbfc815a5ea sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30682024-11-15T20:11:33.125ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39023 method = GET remote_addr = [::1]:36494 req_id = fbd120e0-1e69-4c7c-a220-0ddb7b98ed76 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30692024-11-15T20:11:33.125ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39023 method = GET remote_addr = [::1]:36496 req_id = 86ef0d2e-437e-40d4-a53c-aa3b0b8bee12 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30702024-11-15T20:11:33.125ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39023 method = GET remote_addr = [::1]:36498 req_id = 144838d4-5bbe-4b51-85da-dbf3ddaa87fe sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30712024-11-15T20:11:33.125ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39023 method = GET remote_addr = [::1]:36500 req_id = 9813d3b3-41e8-47ad-944a-cc9fe4cfaad7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30722024-11-15T20:11:33.125ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1240 local_addr = [::1]:39023 method = GET remote_addr = [::1]:36488 req_id = 5c6965ad-95ed-4f6b-9926-f2df10bbca27 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30732024-11-15T20:11:33.126ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1189 local_addr = [::1]:39023 method = GET remote_addr = [::1]:36490 req_id = 1913ace3-df31-42e5-9edd-41d5621e368b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30742024-11-15T20:11:33.126ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1090 local_addr = [::1]:39023 method = GET remote_addr = [::1]:36492 req_id = e4f149ec-0fc9-4baf-9182-bfbfc815a5ea response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30752024-11-15T20:11:33.126ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 1013 local_addr = [::1]:39023 method = GET remote_addr = [::1]:36494 req_id = fbd120e0-1e69-4c7c-a220-0ddb7b98ed76 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30762024-11-15T20:11:33.126ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 908 local_addr = [::1]:39023 method = GET remote_addr = [::1]:36496 req_id = 86ef0d2e-437e-40d4-a53c-aa3b0b8bee12 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30772024-11-15T20:11:33.126ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 811 local_addr = [::1]:39023 method = GET remote_addr = [::1]:36498 req_id = 144838d4-5bbe-4b51-85da-dbf3ddaa87fe response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30782024-11-15T20:11:33.126ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 724 local_addr = [::1]:39023 method = GET remote_addr = [::1]:36500 req_id = 9813d3b3-41e8-47ad-944a-cc9fe4cfaad7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
30792024-11-15T20:11:33.137ZINFOtest_mgs_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 11003f7a-2033-4d74-8c8e-10a3d3bb41d8
30802024-11-15T20:11:33.137ZDEBGtest_mgs_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 11003f7a-2033-4d74-8c8e-10a3d3bb41d8 time_started = 2024-11-15 20:11:31.299498 UTC
30812024-11-15T20:11:33.140ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.365905109s iteration = 4 status = {"collection_id":"11003f7a-2033-4d74-8c8e-10a3d3bb41d8","time_done":"2024-11-15 20:11:31.712096 UTC","time_started":"2024-11-15 20:11:31.299498 UTC"}
30822024-11-15T20:11:33.140ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
30832024-11-15T20:11:33.140ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 37.112\u{b5}s iteration = 2 status = {"error":"task disabled"}
30842024-11-15T20:11:33.150ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativei64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30852024-11-15T20:11:33.163ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativeu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30862024-11-15T20:11:33.206ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:37069/metrics/producers
30872024-11-15T20:11:33.206ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30882024-11-15T20:11:33.206ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44686
30892024-11-15T20:11:33.207ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44686 req_id = 3e1b0fdd-26f6-4f43-91c2-0e8cb1511ddb uri = /metrics/producers
30902024-11-15T20:11:33.208ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44686 req_id = 3e1b0fdd-26f6-4f43-91c2-0e8cb1511ddb resource = Database uri = /metrics/producers
30912024-11-15T20:11:33.208ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44686 req_id = 3e1b0fdd-26f6-4f43-91c2-0e8cb1511ddb roles = RoleSet { roles: {} } uri = /metrics/producers
30922024-11-15T20:11:33.209ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44686 req_id = 3e1b0fdd-26f6-4f43-91c2-0e8cb1511ddb resource = Database result = Ok(()) uri = /metrics/producers
30932024-11-15T20:11:33.214ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 6510 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44686 req_id = 3e1b0fdd-26f6-4f43-91c2-0e8cb1511ddb response_code = 503 uri = /metrics/producers
30942024-11-15T20:11:33.215ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:37069/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "3e1b0fdd-26f6-4f43-91c2-0e8cb1511ddb", "content-length": "133", "date": "Fri, 15 Nov 2024 20:11:32 GMT"} })
30952024-11-15T20:11:33.216ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 705.445498ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"3e1b0fdd-26f6-4f43-91c2-0e8cb1511ddb\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 15 Nov 2024 20:11:32 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"3e1b0fdd-26f6-4f43-91c2-0e8cb1511ddb\\" }" file = oximeter/producer/src/lib.rs:420
30962024-11-15T20:11:33.227ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30972024-11-15T20:11:33.246ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Array types cannot be Nullable, see\n * https://clickhouse.com/docs/en/sql-reference/data-types/nullable\n * for more details.\n *\n * This means we need to use empty arrays to indicate a missing value. This\n * is unfortunate, and at this point relies on the fact that an\n * `oximeter::Histogram` cannot have zero bins. If that changes, we'll need\n * to figure out another way to represent missing samples here.\n */\n bins Array(Int8),\n counts Array(UInt64),\n min Int8,\n max Int8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30982024-11-15T20:11:33.262ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt8),\n counts Array(UInt64),\n min UInt8,\n max UInt8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30992024-11-15T20:11:33.278ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int16),\n counts Array(UInt64),\n min Int16,\n max Int16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31002024-11-15T20:11:33.305ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt16),\n counts Array(UInt64),\n min UInt16,\n max UInt16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31012024-11-15T20:11:33.319ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int32),\n counts Array(UInt64),\n min Int32,\n max Int32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31022024-11-15T20:11:33.331ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt32),\n counts Array(UInt64),\n min UInt32,\n max UInt32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31032024-11-15T20:11:33.348ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int64),\n counts Array(UInt64),\n min Int64,\n max Int64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31042024-11-15T20:11:33.360ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt64),\n counts Array(UInt64),\n min UInt64,\n max UInt64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31052024-11-15T20:11:33.371ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float32),\n counts Array(UInt64),\n min Float32,\n max Float32,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31062024-11-15T20:11:33.384ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float64),\n counts Array(UInt64),\n min Float64,\n max Float64,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31072024-11-15T20:11:33.396ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\n/* The field tables store named dimensions of each timeseries.\n *\n * As with the measurement tables, there is one field table for each field data\n * type. Fields are deduplicated by using the "replacing merge tree", though\n * this behavior **must not** be relied upon for query correctness.\n *\n * The index for the fields differs from the measurements, however. Rows are\n * sorted by timeseries name, then field name, field value, and finally\n * timeseries key. This reflects the most common pattern for looking them up:\n * by field name and possibly value, within a timeseries. The resulting keys are\n * usually then used to look up measurements.\n *\n * NOTE: We may want to consider a secondary index on these tables, sorting by\n * timeseries name and then key, since it would improve lookups where one\n * already has the key. Realistically though, these tables are quite small and\n * so performance benefits will be low in absolute terms.\n *\n * TTL: We use a materialized column to expire old field table records. This\n * column is generated automatically by the database whenever a new row is\n * inserted. It cannot be inserted directly, nor is it returned in a `SELECT *`\n * query. Since these tables are `ReplacingMergeTree`s, that means the last\n * record will remain during a deduplication, which will have the last\n * timestamp. ClickHouse will then expire old data for us, similar to the\n * measurement tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.fields_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Bool,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31082024-11-15T20:11:33.409ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31092024-11-15T20:11:33.428ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31102024-11-15T20:11:33.442ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31112024-11-15T20:11:33.463ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31122024-11-15T20:11:33.478ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31132024-11-15T20:11:33.533ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31142024-11-15T20:11:33.547ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31152024-11-15T20:11:33.561ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31162024-11-15T20:11:33.576ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_ipaddr\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value IPv6,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31172024-11-15T20:11:33.607ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value String,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31182024-11-15T20:11:33.620ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_uuid\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UUID,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
31192024-11-15T20:11:33.633ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = \n\n/* The timeseries schema table stores the extracted schema for the samples\n * oximeter collects.\n */\nCREATE TABLE IF NOT EXISTS oximeter.timeseries_schema\n(\n timeseries_name String,\n fields Nested(\n name String,\n type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'IpAddr' = 3,\n 'String' = 4,\n 'Uuid' = 6,\n 'I8' = 7,\n 'U8' = 8,\n 'I16' = 9,\n 'U16' = 10,\n 'I32' = 11,\n 'U32' = 12,\n 'U64' = 13\n ),\n source Enum(\n 'Target' = 1,\n 'Metric' = 2\n )\n ),\n datum_type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'F64' = 3,\n 'String' = 4,\n 'Bytes' = 5,\n 'CumulativeI64' = 6,\n 'CumulativeF64' = 7,\n 'HistogramI64' = 8,\n 'HistogramF64' = 9,\n 'I8' = 10,\n 'U8' = 11,\n 'I16' = 12,\n 'U16' = 13,\n 'I32' = 14,\n 'U32' = 15,\n 'U64' = 16,\n 'F32' = 17,\n 'CumulativeU64' = 18,\n 'CumulativeF32' = 19,\n 'HistogramI8' = 20,\n 'HistogramU8' = 21,\n 'HistogramI16' = 22,\n 'HistogramU16' = 23,\n 'HistogramI32' = 24,\n 'HistogramU32' = 25,\n 'HistogramU64' = 26,\n 'HistogramF32' = 27\n ),\n created DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, fields.name)
31202024-11-15T20:11:33.651ZINFOtest_mgs_metrics (clickhouse-client): inserting current version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:883 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 version = 13
31212024-11-15T20:11:33.655ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = INSERT INTO oximeter.version (*) VALUES (13, now());
31222024-11-15T20:11:33.659ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34767 method = GET path = /info versions = all
31232024-11-15T20:11:33.660ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34767 method = GET path = /producers versions = all
31242024-11-15T20:11:33.660ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34767 method = DELETE path = /producers/{producer_id} versions = all
31252024-11-15T20:11:33.660ZINFOtest_mgs_metrics (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [::1]:34767
31262024-11-15T20:11:33.660ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:34767
31272024-11-15T20:11:33.660ZDEBGtest_mgs_metrics (oximeter): registered USDT probes
31282024-11-15T20:11:33.660ZDEBGtest_mgs_metrics (oximeter): contacting nexus
31292024-11-15T20:11:33.962ZDEBGtest_mgs_metrics (oximeter): client request body = Some(Body) method = POST uri = http://127.0.0.1:37069/metrics/collectors
31302024-11-15T20:11:34.016ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:37069/metrics/producers
31312024-11-15T20:11:34.039ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44688
31322024-11-15T20:11:34.045ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44690
31332024-11-15T20:11:34.045ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44690 req_id = df779f66-3529-4da7-935d-b0c09c22a795 uri = /metrics/producers
31342024-11-15T20:11:34.045ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44690 req_id = df779f66-3529-4da7-935d-b0c09c22a795 resource = Database uri = /metrics/producers
31352024-11-15T20:11:34.045ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44690 req_id = df779f66-3529-4da7-935d-b0c09c22a795 roles = RoleSet { roles: {} } uri = /metrics/producers
31362024-11-15T20:11:34.045ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44690 req_id = df779f66-3529-4da7-935d-b0c09c22a795 resource = Database result = Ok(()) uri = /metrics/producers
31372024-11-15T20:11:34.050ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44688 req_id = 4c6da978-87fb-4acd-ac9d-503369bb4b46 uri = /metrics/collectors
31382024-11-15T20:11:34.050ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44688 req_id = 4c6da978-87fb-4acd-ac9d-503369bb4b46 resource = Database uri = /metrics/collectors
31392024-11-15T20:11:34.050ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44688 req_id = 4c6da978-87fb-4acd-ac9d-503369bb4b46 roles = RoleSet { roles: {} } uri = /metrics/collectors
31402024-11-15T20:11:34.051ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44688 req_id = 4c6da978-87fb-4acd-ac9d-503369bb4b46 resource = Database result = Ok(()) uri = /metrics/collectors
31412024-11-15T20:11:34.053ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 12920 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44690 req_id = df779f66-3529-4da7-935d-b0c09c22a795 response_code = 503 uri = /metrics/producers
31422024-11-15T20:11:34.054ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:37069/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "df779f66-3529-4da7-935d-b0c09c22a795", "content-length": "133", "date": "Fri, 15 Nov 2024 20:11:34 GMT"} })
31432024-11-15T20:11:34.056ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.014103687s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"df779f66-3529-4da7-935d-b0c09c22a795\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 15 Nov 2024 20:11:34 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"df779f66-3529-4da7-935d-b0c09c22a795\\" }" file = oximeter/producer/src/lib.rs:420
31442024-11-15T20:11:34.065ZINFOtest_mgs_metrics (nexus): registered new oximeter metric collection server address = [::1]:34767 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31452024-11-15T20:11:34.070ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 18890 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44688 req_id = 4c6da978-87fb-4acd-ac9d-503369bb4b46 response_code = 204 uri = /metrics/collectors
31462024-11-15T20:11:34.070ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:37069/metrics/collectors", status: 204, headers: {"x-request-id": "4c6da978-87fb-4acd-ac9d-503369bb4b46", "date": "Fri, 15 Nov 2024 20:11:34 GMT"} })
31472024-11-15T20:11:34.070ZINFOtest_mgs_metrics (oximeter): oximeter registered with nexus file = oximeter/collector/src/lib.rs:401 id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31482024-11-15T20:11:34.070ZDEBGtest_mgs_metrics (init_with_steps): Running step start_producer_server
31492024-11-15T20:11:34.070ZDEBGtest_mgs_metrics: Starting test metric Producer Server
31502024-11-15T20:11:34.219ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44692
31512024-11-15T20:11:34.224ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:37069 remote_addr = 127.0.0.1:44694
31522024-11-15T20:11:34.225ZINFOtest_mgs_metrics (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:772
31532024-11-15T20:11:34.225ZDEBGtest_mgs_metrics (oximeter): client request body = None method = GET uri = http://127.0.0.1:37069/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31542024-11-15T20:11:34.226ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44692 req_id = f9a8e216-d09e-4925-bf99-e7289b7a859d uri = /metrics/producers
31552024-11-15T20:11:34.227ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44694 req_id = b04479c3-90cc-4bea-a9ac-b250f17d66d7 uri = /metrics/producers
31562024-11-15T20:11:34.228ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 6cc8b7d1-494f-4f21-88d2-3202bf80b977 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31572024-11-15T20:11:34.228ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44692 req_id = f9a8e216-d09e-4925-bf99-e7289b7a859d resource = Database uri = /metrics/producers
31582024-11-15T20:11:34.228ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44692 req_id = f9a8e216-d09e-4925-bf99-e7289b7a859d roles = RoleSet { roles: {} } uri = /metrics/producers
31592024-11-15T20:11:34.233ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44692 req_id = f9a8e216-d09e-4925-bf99-e7289b7a859d resource = Database result = Ok(()) uri = /metrics/producers
31602024-11-15T20:11:34.233ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44694 req_id = b04479c3-90cc-4bea-a9ac-b250f17d66d7 resource = Database uri = /metrics/producers
31612024-11-15T20:11:34.235ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44694 req_id = b04479c3-90cc-4bea-a9ac-b250f17d66d7 roles = RoleSet { roles: {} } uri = /metrics/producers
31622024-11-15T20:11:34.235ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44694 req_id = b04479c3-90cc-4bea-a9ac-b250f17d66d7 resource = Database result = Ok(()) uri = /metrics/producers
31632024-11-15T20:11:34.238ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 6cc8b7d1-494f-4f21-88d2-3202bf80b977 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31642024-11-15T20:11:34.238ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 6cc8b7d1-494f-4f21-88d2-3202bf80b977 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31652024-11-15T20:11:34.239ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 6cc8b7d1-494f-4f21-88d2-3202bf80b977 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31662024-11-15T20:11:34.254ZINFOtest_mgs_metrics (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31672024-11-15T20:11:34.254ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 27807 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44692 req_id = f9a8e216-d09e-4925-bf99-e7289b7a859d response_code = 201 uri = /metrics/producers
31682024-11-15T20:11:34.254ZINFOtest_mgs_metrics (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
31692024-11-15T20:11:34.254ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 27954 local_addr = 127.0.0.1:37069 method = POST remote_addr = 127.0.0.1:44694 req_id = b04479c3-90cc-4bea-a9ac-b250f17d66d7 response_code = 201 uri = /metrics/producers
31702024-11-15T20:11:34.261ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 34613 local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 6cc8b7d1-494f-4f21-88d2-3202bf80b977 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31712024-11-15T20:11:34.262ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:37069/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cc8b7d1-494f-4f21-88d2-3202bf80b977", "content-length": "412", "date": "Fri, 15 Nov 2024 20:11:34 GMT"} })
31722024-11-15T20:11:34.262ZDEBGtest_mgs_metrics (oximeter): client request body = None method = GET uri = http://127.0.0.1:37069/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31732024-11-15T20:11:34.263ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 071c1868-d04b-42c3-8080-0e67062d69d0 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31742024-11-15T20:11:34.264ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 071c1868-d04b-42c3-8080-0e67062d69d0 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31752024-11-15T20:11:34.264ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 071c1868-d04b-42c3-8080-0e67062d69d0 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31762024-11-15T20:11:34.264ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 071c1868-d04b-42c3-8080-0e67062d69d0 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31772024-11-15T20:11:34.267ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 4684 local_addr = 127.0.0.1:37069 method = GET remote_addr = 127.0.0.1:44688 req_id = 071c1868-d04b-42c3-8080-0e67062d69d0 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D
31782024-11-15T20:11:34.267ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:37069/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhNjQ1OGI3ZC04N2MzLTQ0ODMtYmU5Ni04NTRkODE0YzIwZGUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "071c1868-d04b-42c3-8080-0e67062d69d0", "content-length": "29", "date": "Fri, 15 Nov 2024 20:11:34 GMT"} })
31792024-11-15T20:11:34.268ZDEBGtest_mgs_metrics (oximeter-agent): registered new metric producer address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
31802024-11-15T20:11:34.268ZDEBGtest_mgs_metrics (oximeter-agent): registered new metric producer address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31812024-11-15T20:11:34.268ZINFOtest_mgs_metrics (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:820 n_current_tasks = 2 n_pruned_tasks = 0
31822024-11-15T20:11:34.304ZDEBGtest_mgs_metrics (collection-task): starting oximeter collection task address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 interval = 10s producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
31832024-11-15T20:11:34.339ZDEBGtest_mgs_metrics (collection-task): starting oximeter collection task address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 interval = 1s producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31842024-11-15T20:11:34.339ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
31852024-11-15T20:11:34.340ZDEBGtest_mgs_metrics (collection-task): collecting from producer address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
31862024-11-15T20:11:34.340ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31872024-11-15T20:11:34.340ZDEBGtest_mgs_metrics (collection-task): collecting from producer address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31882024-11-15T20:11:34.343ZDEBGtest_mgs_metrics (collection-task): collected results from producer address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 2 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
31892024-11-15T20:11:34.343ZDEBGtest_mgs_metrics (collection-task): collecting from producer address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
31902024-11-15T20:11:34.344ZDEBGtest_mgs_metrics (collection-task): collected results from producer address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31912024-11-15T20:11:34.344ZDEBGtest_mgs_metrics (collection-task): collecting from producer address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31922024-11-15T20:11:34.344ZDEBGtest_mgs_metrics (results-sink): inserting 0 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31932024-11-15T20:11:34.344ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 0 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
31942024-11-15T20:11:34.344ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31952024-11-15T20:11:34.344ZDEBGtest_mgs_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31962024-11-15T20:11:34.344ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
31972024-11-15T20:11:34.345ZDEBGtest_mgs_metrics (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
31982024-11-15T20:11:34.345ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
31992024-11-15T20:11:34.346ZDEBGtest_mgs_metrics (collection-task): collected results from producer address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 2 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
32002024-11-15T20:11:34.347ZDEBGtest_mgs_metrics (collection-task): collected results from producer address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
32012024-11-15T20:11:34.348ZTRACtest_mgs_metrics (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
32022024-11-15T20:11:34.348ZDEBGtest_mgs_metrics (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 schema = TimeseriesSchema { timeseries_name: TimeseriesName("integration_target:integration_metric"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "metric_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "target_name", field_type: String, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2024-11-15T20:11:34.344849619Z } timeseries_name = integration_target:integration_metric
32032024-11-15T20:11:34.348ZDEBGtest_mgs_metrics (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
32042024-11-15T20:11:34.348ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_columns = 6 n_rows = 1 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
32052024-11-15T20:11:34.356ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
32062024-11-15T20:11:34.363ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_rows = 2 table_name = fields_string
32072024-11-15T20:11:34.363ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
32082024-11-15T20:11:34.367ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_rows = 1 table_name = measurements_i64
32092024-11-15T20:11:34.369ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32102024-11-15T20:11:34.369ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
32112024-11-15T20:11:34.369ZDEBGtest_mgs_metrics (collection-task): collecting from producer address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
32122024-11-15T20:11:34.369ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
32132024-11-15T20:11:34.369ZDEBGtest_mgs_metrics (collection-task): collecting from producer address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
32142024-11-15T20:11:34.370ZDEBGtest_mgs_metrics (collection-task): collected results from producer address = [::1]:42841 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 2 producer_id = 4434f2ae-de14-477f-a5d3-d8042811b01e
32152024-11-15T20:11:34.371ZDEBGtest_mgs_metrics (collection-task): collected results from producer address = [::1]:35621 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
32162024-11-15T20:11:34.371ZDEBGtest_mgs_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32172024-11-15T20:11:34.371ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
32182024-11-15T20:11:34.372ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
32192024-11-15T20:11:34.378ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_rows = 2 table_name = fields_string
32202024-11-15T20:11:34.379ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
32212024-11-15T20:11:34.390ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_rows = 1 table_name = measurements_i64
32222024-11-15T20:11:34.391ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32232024-11-15T20:11:34.391ZDEBGtest_mgs_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32242024-11-15T20:11:34.392ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67
32252024-11-15T20:11:34.392ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
32262024-11-15T20:11:34.409ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_rows = 2 table_name = fields_string
32272024-11-15T20:11:34.411ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
32282024-11-15T20:11:34.423ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 0d57286b-6ee4-47ff-a3b4-adda8cca5e67 n_rows = 1 table_name = measurements_i64
32292024-11-15T20:11:34.423ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32302024-11-15T20:11:34.423ZINFOtest_mgs_metrics (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:38265/v1/system/timeseries/query
32312024-11-15T20:11:34.427ZINFOtest_mgs_metrics (dropshot_external): accepted connection local_addr = 127.0.0.1:38265 remote_addr = 127.0.0.1:51052
32322024-11-15T20:11:34.428ZTRACtest_mgs_metrics (dropshot_external): incoming request local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d uri = /v1/system/timeseries/query
32332024-11-15T20:11:34.428ZTRACtest_mgs_metrics (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d uri = /v1/system/timeseries/query
32342024-11-15T20:11:34.428ZDEBGtest_mgs_metrics (dropshot_external): looking up silo for user local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/timeseries/query
32352024-11-15T20:11:34.428ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32362024-11-15T20:11:34.428ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32372024-11-15T20:11:34.429ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32382024-11-15T20:11:34.431ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32392024-11-15T20:11:34.432ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32402024-11-15T20:11:34.432ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32412024-11-15T20:11:34.436ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
32422024-11-15T20:11:34.437ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32432024-11-15T20:11:34.437ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32442024-11-15T20:11:34.437ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32452024-11-15T20:11:34.437ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32462024-11-15T20:11:34.439ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32472024-11-15T20:11:34.439ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32482024-11-15T20:11:34.439ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32492024-11-15T20:11:34.439ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32502024-11-15T20:11:34.444ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32512024-11-15T20:11:34.447ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
32522024-11-15T20:11:34.447ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32532024-11-15T20:11:34.447ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32542024-11-15T20:11:34.448ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32552024-11-15T20:11:34.450ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
32562024-11-15T20:11:34.450ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32572024-11-15T20:11:34.450ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32582024-11-15T20:11:34.450ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32592024-11-15T20:11:34.450ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32602024-11-15T20:11:34.452ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32612024-11-15T20:11:34.452ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32622024-11-15T20:11:34.452ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
32632024-11-15T20:11:34.452ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32642024-11-15T20:11:34.454ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32652024-11-15T20:11:34.457ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
32662024-11-15T20:11:34.457ZTRACtest_mgs_metrics (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d uri = /v1/system/timeseries/query
32672024-11-15T20:11:34.457ZTRACtest_mgs_metrics (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d resource = Fleet uri = /v1/system/timeseries/query
32682024-11-15T20:11:34.457ZTRACtest_mgs_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/timeseries/query
32692024-11-15T20:11:34.457ZTRACtest_mgs_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d resource = Database uri = /v1/system/timeseries/query
32702024-11-15T20:11:34.457ZDEBGtest_mgs_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d roles = RoleSet { roles: {} } uri = /v1/system/timeseries/query
32712024-11-15T20:11:34.458ZDEBGtest_mgs_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d resource = Database result = Ok(()) uri = /v1/system/timeseries/query
32722024-11-15T20:11:34.470ZDEBGtest_mgs_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/timeseries/query
32732024-11-15T20:11:34.475ZDEBGtest_mgs_metrics (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d resource = Fleet result = Ok(()) uri = /v1/system/timeseries/query
32742024-11-15T20:11:34.475ZDEBGtest_mgs_metrics (clickhouse-client): parsed OxQL query id = cf71258c-2316-4088-9145-f927ca712583 parsed_query = Query { parsed: Query { ops: [Basic(Get(TimeseriesName("hardware_component:temperature")))] }, end_time: 2024-11-15T20:11:34.475772033Z } query = get hardware_component:temperature query_id = d24373cd-9ce0-465b-a535-4c7a6979d0e8
32752024-11-15T20:11:34.476ZDEBGtest_mgs_metrics (clickhouse-client): retrieving timeseries schema from database id = cf71258c-2316-4088-9145-f927ca712583
32762024-11-15T20:11:34.476ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query id = cf71258c-2316-4088-9145-f927ca712583 sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
32772024-11-15T20:11:34.479ZTRACtest_mgs_metrics (clickhouse-client): retrieved new timeseries schema id = cf71258c-2316-4088-9145-f927ca712583 n_schema = 1
32782024-11-15T20:11:34.480ZINFOtest_mgs_metrics (dropshot_external): request completed error_message_external = Timeseries not found for: hardware_component:temperature error_message_internal = Timeseries not found for: hardware_component:temperature latency_us = 52728 local_addr = 127.0.0.1:38265 method = POST remote_addr = 127.0.0.1:51052 req_id = 2b2d395e-3586-4825-8384-f5935e8a8e9d response_code = 400 uri = /v1/system/timeseries/query
32792024-11-15T20:11:34.481ZINFOtest_mgs_metrics (external client test context): client received response status = 400