Max level shown:
12024-12-20T21:46:15.790ZDEBGtest_mgs_metrics (init_with_steps): Running step start_crdb
22024-12-20T21:46:15.790ZDEBGtest_mgs_metrics: Starting CRDB
32024-12-20T21:46:15.791ZINFOtest_mgs_metrics: cockroach temporary directory: /var/tmp/omicron_tmp/.tmpjWApb1
42024-12-20T21:46:15.792ZINFOtest_mgs_metrics: cockroach: copying from seed tarball (/var/tmp/omicron_tmp/crdb-base-build/345df35a854b98fb9ca917bc1faf83f3d09ca36aa592165a23be654a8eec19c1.tar) to storage directory (/var/tmp/omicron_tmp/.tmpjWApb1/data)
52024-12-20T21:46:15.844ZINFOtest_mgs_metrics: cockroach command line: cockroach start-single-node --insecure --http-addr=:0 --store=path=/var/tmp/omicron_tmp/.tmpjWApb1/data,ballast-size=0 --listen-addr [::1]:0 --listening-url-file /var/tmp/omicron_tmp/.tmpjWApb1/listen-url
62024-12-20T21:46:15.844ZINFOtest_mgs_metrics: cockroach environment: BUILDOMAT_JOB_ID=01JFJYHX8JX8JPA7F6XG85X91S BUILDOMAT_TASK_ID=4 CARGO=/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/bin/cargo CARGO_HOME=/home/build/.cargo CARGO_INCREMENTAL=0 CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus CARGO_PKG_AUTHORS= CARGO_PKG_DESCRIPTION= CARGO_PKG_HOMEPAGE= CARGO_PKG_LICENSE=MPL-2.0 CARGO_PKG_LICENSE_FILE= CARGO_PKG_NAME=omicron-nexus CARGO_PKG_REPOSITORY= CARGO_PKG_RUST_VERSION= CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= CARGO_TERM_COLOR=always CI=true CRDB_SEED_TAR=/var/tmp/omicron_tmp/crdb-base-build/345df35a854b98fb9ca917bc1faf83f3d09ca36aa592165a23be654a8eec19c1.tar GITHUB_BRANCH=gjcolombo/ingest-propolis-ensure-api GITHUB_REF=refs/heads/gjcolombo/ingest-propolis-ensure-api GITHUB_REPOSITORY=oxidecomputer/omicron GITHUB_SHA=f0c692d7aa81d9251d0caeb6ec1c2c943ee12d47 GOTRACEBACK=crash HOME=/home/build LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-0605127ef2db4010/out:/work/oxidecomputer/omicron/target/debug/build/blake3-1e464eee91b26c77/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-70e33ff4b45d8dc0/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-6a6e82c218d03cb7/out/build:/work/oxidecomputer/omicron/target/debug/build/libz-sys-33d79b7b3a6a5533/out/lib:/work/oxidecomputer/omicron/target/debug/build/ring-84313340968b144b/out:/work/oxidecomputer/omicron/target/debug/build/ring-ae424a22a6f8daa6/out:/work/oxidecomputer/omicron/target/debug/build/tofino-b9675a1b9b2f961b/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib LOGNAME=build NEXTEST=1 NEXTEST_BIN_EXE_nexus=/work/oxidecomputer/omicron/target/debug/nexus NEXTEST_BIN_EXE_schema-updater=/work/oxidecomputer/omicron/target/debug/schema-updater NEXTEST_EXECUTION_MODE=process-per-test NEXTEST_LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-0605127ef2db4010/out:/work/oxidecomputer/omicron/target/debug/build/blake3-1e464eee91b26c77/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-70e33ff4b45d8dc0/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-6a6e82c218d03cb7/out/build:/work/oxidecomputer/omicron/target/debug/build/libz-sys-33d79b7b3a6a5533/out/lib:/work/oxidecomputer/omicron/target/debug/build/ring-84313340968b144b/out:/work/oxidecomputer/omicron/target/debug/build/ring-ae424a22a6f8daa6/out:/work/oxidecomputer/omicron/target/debug/build/tofino-b9675a1b9b2f961b/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/build/.rustup/toolchains/1.81.0-x86_64-unknown-linux-gnu/lib NEXTEST_RUN_ID=44fc9d97-ee88-4021-8ba1-c680b8c11e6b OUT_DIR=/work/oxidecomputer/omicron/target/debug/build/omicron-nexus-4fb98a22e58c957e/out PATH=/work/oxidecomputer/omicron/out/mgd/root/opt/oxide/mgd/bin:/work/oxidecomputer/omicron/out/dendrite-stub/bin:/work/oxidecomputer/omicron/out/clickhouse:/work/oxidecomputer/omicron/out/cockroachdb/bin:/home/build/.cargo/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/ooce/bin:/opt/ooce/sbin PWD=/work/oxidecomputer/omicron RUSTC_BOOTSTRAP=1 RUSTDOCFLAGS=--document-private-items -D warnings RUSTFLAGS=-D warnings RUSTUP_HOME=/home/build/.rustup RUSTUP_TOOLCHAIN=1.81.0-x86_64-unknown-linux-gnu RUST_BACKTRACE=1 RUST_RECURSION_COUNT=1 SHLVL=1 SSL_CERT_DIR=/usr/lib/ssl/certs SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt TMPDIR=/var/tmp/omicron_tmp TZ=UTC USER=build __NEXTEST_ATTEMPT=1
72024-12-20T21:46:17.168ZINFOtest_mgs_metrics: cockroach pid: 163612
82024-12-20T21:46:17.172ZINFOtest_mgs_metrics: cockroach listen URL: postgresql://root@[::1]:43309/omicron?sslmode=disable
92024-12-20T21:46:17.172ZDEBGtest_mgs_metrics (init_with_steps): Running step start_clickhouse
102024-12-20T21:46:17.172ZDEBGtest_mgs_metrics: Starting Clickhouse
112024-12-20T21:46:17.743ZDEBGtest_mgs_metrics (init_with_steps): Running step start_gateway_switch0
122024-12-20T21:46:17.743ZDEBGtest_mgs_metrics: Starting Management Gateway
132024-12-20T21:46:17.871ZDEBGtest_mgs_metrics (init_with_steps): Running step start_gateway_switch1
142024-12-20T21:46:17.874ZDEBGtest_mgs_metrics: Starting Management Gateway
152024-12-20T21:46:18.016ZDEBGtest_mgs_metrics (init_with_steps): Running step start_dendrite_switch0
162024-12-20T21:46:18.016ZDEBGtest_mgs_metrics: Starting Dendrite for switch0
172024-12-20T21:46:18.098ZDEBGtest_mgs_metrics (init_with_steps): Running step start_dendrite_switch1
182024-12-20T21:46:18.098ZDEBGtest_mgs_metrics: Starting Dendrite for switch1
192024-12-20T21:46:18.131ZDEBGtest_mgs_metrics (init_with_steps): Running step start_mgd_switch0
202024-12-20T21:46:18.131ZDEBGtest_mgs_metrics: Starting mgd for switch0
212024-12-20T21:46:18.195ZDEBGtest_mgs_metrics: mgd port is 35923
222024-12-20T21:46:18.195ZDEBGtest_mgs_metrics (init_with_steps): Running step start_mgd_switch1
232024-12-20T21:46:18.197ZDEBGtest_mgs_metrics: Starting mgd for switch1
242024-12-20T21:46:18.230ZDEBGtest_mgs_metrics: mgd port is 37229
252024-12-20T21:46:18.231ZDEBGtest_mgs_metrics (init_with_steps): Running step record_switch_dns
262024-12-20T21:46:18.231ZDEBGtest_mgs_metrics: Recording DNS for the switch zones
272024-12-20T21:46:18.231ZDEBGtest_mgs_metrics (init_with_steps): Running step start_internal_dns
282024-12-20T21:46:18.231ZINFOtest_mgs_metrics (store): opening sled database path = /var/tmp/omicron_tmp/.tmpxgLtnL
292024-12-20T21:46:18.311ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
302024-12-20T21:46:18.312ZINFOtest_mgs_metrics (store): pruning trees for generations older than 0 keep = 3
312024-12-20T21:46:18.312ZINFOtest_mgs_metrics (dns): DNS server bound to address kind = dns local_address = [::1]:45014
322024-12-20T21:46:18.312ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:34419 method = GET path = /config versions = all
332024-12-20T21:46:18.312ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:34419 method = PUT path = /config versions = all
342024-12-20T21:46:18.312ZINFOtest_mgs_metrics (http): listening kind = dns local_addr = [::1]:34419
352024-12-20T21:46:18.312ZDEBGtest_mgs_metrics (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:34419
362024-12-20T21:46:18.312ZDEBGtest_mgs_metrics (init_with_steps): Running step start_external_dns
372024-12-20T21:46:18.334ZINFOtest_mgs_metrics (store): opening sled database path = /var/tmp/omicron_tmp/.tmpFo6g5f
382024-12-20T21:46:18.352ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
392024-12-20T21:46:18.352ZINFOtest_mgs_metrics (store): pruning trees for generations older than 0 keep = 3
402024-12-20T21:46:18.352ZINFOtest_mgs_metrics (dns): DNS server bound to address kind = dns local_address = [::1]:40829
412024-12-20T21:46:18.352ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:40731 method = GET path = /config versions = all
422024-12-20T21:46:18.352ZDEBGtest_mgs_metrics (http): registered endpoint kind = dns local_addr = [::1]:40731 method = PUT path = /config versions = all
432024-12-20T21:46:18.352ZINFOtest_mgs_metrics (http): listening kind = dns local_addr = [::1]:40731
442024-12-20T21:46:18.352ZDEBGtest_mgs_metrics (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:40731
452024-12-20T21:46:18.352ZDEBGtest_mgs_metrics (init_with_steps): Running step start_nexus_internal
462024-12-20T21:46:18.355ZDEBGtest_mgs_metrics: Starting Nexus (internal API)
472024-12-20T21:46:18.356ZINFOtest_mgs_metrics: setting up nexus server
482024-12-20T21:46:18.356ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Action
492024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AnyActor
502024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AuthenticatedActor
512024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = BlueprintConfig
522024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Database
532024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DnsConfig
542024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Fleet
552024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Inventory
562024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IpPoolList
572024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ConsoleSessionList
582024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAuthRequestList
592024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloCertificateList
602024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloIdentityProviderList
612024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloUserList
622024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Project
632024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Disk
642024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Snapshot
652024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ProjectImage
662024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Instance
672024-12-20T21:46:18.357ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IpPool
682024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InstanceNetworkInterface
692024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Vpc
702024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = VpcRouter
712024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGateway
722024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGatewayIpPool
732024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = InternetGatewayIpAddress
742024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = RouterRoute
752024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = VpcSubnet
762024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = FloatingIp
772024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Image
782024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloImage
792024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = AddressLot
802024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Blueprint
812024-12-20T21:46:18.359ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = LoopbackAddress
822024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Certificate
832024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = ConsoleSession
842024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAuthRequest
852024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = DeviceAccessToken
862024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = PhysicalDisk
872024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Rack
882024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = RoleBuiltin
892024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SshKey
902024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Silo
912024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloUser
922024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SiloGroup
932024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = IdentityProvider
942024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = SamlIdentityProvider
952024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Sled
962024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = TufRepo
972024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = TufArtifact
982024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Zpool
992024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = Service
1002024-12-20T21:46:18.360ZINFOtest_mgs_metrics (ServerContext): registering Oso class class = UserBuiltin
1012024-12-20T21:46:18.361ZINFOtest_mgs_metrics (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
1022024-12-20T21:46:18.396ZINFOtest_mgs_metrics (ServerContext): Setting up resolver using DNS address: [::1]:45014
1032024-12-20T21:46:18.396ZINFOtest_mgs_metrics (DnsResolver): new DNS resolver addresses = [[::1]:45014]
1042024-12-20T21:46:18.397ZINFOtest_mgs_metrics (ServerContext): Setting up qorb database pool from a single host url = PostgresConfigWithUrl {\n url_raw: "postgresql://root@[::1]:43309/omicron?sslmode=disable",\n config: Config {\n user: Some(\n "root",\n ),\n password: None,\n dbname: Some(\n "omicron",\n ),\n options: None,\n application_name: None,\n ssl_mode: Disable,\n host: [\n Tcp(\n "::1",\n ),\n ],\n hostaddr: [],\n port: [\n 43309,\n ],\n connect_timeout: None,\n tcp_user_timeout: None,\n keepalives: true,\n keepalives_idle: 7200s,\n keepalives_interval: None,\n keepalives_retries: None,\n target_session_attrs: Any,\n channel_binding: Prefer,\n },\n}
1052024-12-20T21:46:18.397ZDEBGtest_mgs_metrics (ServerContext): registered USDT probes
1062024-12-20T21:46:18.498ZINFOtest_mgs_metrics (nexus): Database schema version is up to date desired_version = 117.0.0 found_version = 117.0.0
1072024-12-20T21:46:18.701ZDEBGtest_mgs_metrics (clickhouse-client): registered USDT probes id = e30132f4-6bad-4aa8-9298-df473cc1db82
1082024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /artifacts/{kind}/{name}/{version} versions = all
1092024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /bgtasks versions = all
1102024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /bgtasks/activate versions = all
1112024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /bgtasks/view/{bgtask_name} versions = all
1122024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /clickhouse/policy versions = all
1132024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /clickhouse/policy versions = all
1142024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
1152024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
1162024-12-20T21:46:18.709ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
1172024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1182024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1192024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /demo-saga versions = all
1202024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1212024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /deployment/blueprints/all versions = all
1222024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1232024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1242024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /deployment/blueprints/import versions = all
1252024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /deployment/blueprints/regenerate versions = all
1262024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /deployment/blueprints/target versions = all
1272024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /deployment/blueprints/target versions = all
1282024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = PUT path = /deployment/blueprints/target/enabled versions = all
1292024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1302024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = PUT path = /disks/{disk_id} versions = all
1312024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /instances/{instance_id}/migrate versions = all
1322024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /metrics/collectors versions = all
1332024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1342024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /metrics/producers versions = all
1352024-12-20T21:46:18.710ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1362024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /physical-disk/expunge versions = all
1372024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /probes/{sled} versions = all
1382024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1392024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /sagas versions = all
1402024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /sagas/{saga_id} versions = all
1412024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /sled-agents/{sled_id} versions = all
1422024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /sled-agents/{sled_id} versions = all
1432024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1442024-12-20T21:46:18.711ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /sleds/add versions = all
1452024-12-20T21:46:18.714ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /sleds/expunge versions = all
1462024-12-20T21:46:18.714ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = GET path = /sleds/uninitialized versions = all
1472024-12-20T21:46:18.714ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = PUT path = /switch/{switch_id} versions = all
1482024-12-20T21:46:18.714ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = PUT path = /vmms/{propolis_id} versions = all
1492024-12-20T21:46:18.714ZDEBGtest_mgs_metrics (dropshot_internal): registered endpoint local_addr = 127.0.0.1:38423 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1502024-12-20T21:46:18.714ZINFOtest_mgs_metrics (dropshot_internal): listening local_addr = 127.0.0.1:38423
1512024-12-20T21:46:18.714ZDEBGtest_mgs_metrics (dropshot_internal): successfully registered DTrace USDT probes local_addr = 127.0.0.1:38423
1522024-12-20T21:46:18.714ZINFOtest_mgs_metrics (SEC): SEC running sec_id = cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176
1532024-12-20T21:46:18.714ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1542024-12-20T21:46:18.714ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1552024-12-20T21:46:18.714ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1562024-12-20T21:46:18.715ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in users
1572024-12-20T21:46:18.715ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1582024-12-20T21:46:18.715ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1592024-12-20T21:46:18.715ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1602024-12-20T21:46:18.718ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 45 a9 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 E............_cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:49970 req_id = b4d22580-8a41-4fa7-92d0-406f47d86f4f
1612024-12-20T21:46:18.720ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17833,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49970 req_id = b4d22580-8a41-4fa7-92d0-406f47d86f4f
1622024-12-20T21:46:18.720ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17833, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_crucible-pantry._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 16, 95, 99, 114, 117, 99, 105, 98, 108, 101, 45, 112, 97, 110, 116, 114, 121, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_crucible-pantry._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49970 req_id = b4d22580-8a41-4fa7-92d0-406f47d86f4f
1632024-12-20T21:46:18.762ZINFOtest_mgs_metrics (DataLoader): created 5 built-in users
1642024-12-20T21:46:18.763ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1652024-12-20T21:46:18.763ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1662024-12-20T21:46:18.765ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1672024-12-20T21:46:18.765ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in roles
1682024-12-20T21:46:18.765ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1692024-12-20T21:46:18.766ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1702024-12-20T21:46:18.766ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1712024-12-20T21:46:18.811ZINFOtest_mgs_metrics (DataLoader): created 10 built-in roles
1722024-12-20T21:46:18.811ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1732024-12-20T21:46:18.811ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1742024-12-20T21:46:18.817ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1752024-12-20T21:46:18.818ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in role assignments
1762024-12-20T21:46:18.818ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1772024-12-20T21:46:18.818ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1782024-12-20T21:46:18.818ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1792024-12-20T21:46:19.408ZINFOtest_mgs_metrics (DataLoader): created 4 built-in role assignments
1802024-12-20T21:46:19.408ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1812024-12-20T21:46:19.408ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1822024-12-20T21:46:19.408ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1832024-12-20T21:46:19.408ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in silos
1842024-12-20T21:46:19.408ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1852024-12-20T21:46:19.408ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1862024-12-20T21:46:19.408ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1872024-12-20T21:46:19.578ZINFOtest_mgs_metrics (DataLoader): created 2 built-in silos
1882024-12-20T21:46:19.578ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1892024-12-20T21:46:19.579ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1902024-12-20T21:46:19.579ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1912024-12-20T21:46:19.617ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1922024-12-20T21:46:19.617ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1932024-12-20T21:46:19.619ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1942024-12-20T21:46:19.619ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in projects
1952024-12-20T21:46:19.619ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1962024-12-20T21:46:19.619ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
1972024-12-20T21:46:19.620ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1982024-12-20T21:46:19.623ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
1992024-12-20T21:46:19.623ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2002024-12-20T21:46:19.623ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2012024-12-20T21:46:19.623ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2022024-12-20T21:46:19.623ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2032024-12-20T21:46:19.704ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2042024-12-20T21:46:19.704ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2052024-12-20T21:46:19.704ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2062024-12-20T21:46:19.705ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2072024-12-20T21:46:19.709ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2082024-12-20T21:46:19.712ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2092024-12-20T21:46:19.713ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2102024-12-20T21:46:19.713ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2112024-12-20T21:46:19.713ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2122024-12-20T21:46:19.713ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2132024-12-20T21:46:19.714ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2142024-12-20T21:46:19.716ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2152024-12-20T21:46:19.716ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2162024-12-20T21:46:19.716ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2172024-12-20T21:46:19.719ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2182024-12-20T21:46:19.720ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2192024-12-20T21:46:19.723ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2202024-12-20T21:46:19.723ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2212024-12-20T21:46:19.723ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2222024-12-20T21:46:19.724ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2232024-12-20T21:46:19.789ZINFOtest_mgs_metrics (DataLoader): created built-in services project
2242024-12-20T21:46:19.789ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2252024-12-20T21:46:19.790ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2262024-12-20T21:46:19.790ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2272024-12-20T21:46:19.790ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPCs
2282024-12-20T21:46:19.790ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2292024-12-20T21:46:19.790ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2302024-12-20T21:46:19.790ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2312024-12-20T21:46:19.808ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2322024-12-20T21:46:19.808ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2332024-12-20T21:46:19.809ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2342024-12-20T21:46:19.811ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2352024-12-20T21:46:19.811ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2362024-12-20T21:46:19.811ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2372024-12-20T21:46:19.811ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2382024-12-20T21:46:19.813ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2392024-12-20T21:46:19.815ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2402024-12-20T21:46:19.815ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2412024-12-20T21:46:19.815ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2422024-12-20T21:46:19.815ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2432024-12-20T21:46:19.817ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2442024-12-20T21:46:19.818ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2452024-12-20T21:46:19.818ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2462024-12-20T21:46:19.818ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2472024-12-20T21:46:19.819ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2482024-12-20T21:46:19.824ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2492024-12-20T21:46:19.824ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Project" using built-in user
2502024-12-20T21:46:19.824ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2512024-12-20T21:46:19.824ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2522024-12-20T21:46:19.824ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2532024-12-20T21:46:19.825ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2542024-12-20T21:46:19.825ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2552024-12-20T21:46:19.829ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2562024-12-20T21:46:19.829ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2572024-12-20T21:46:19.829ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2582024-12-20T21:46:19.830ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2592024-12-20T21:46:19.837ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2602024-12-20T21:46:19.837ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2612024-12-20T21:46:19.837ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2622024-12-20T21:46:19.838ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2632024-12-20T21:46:19.842ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2642024-12-20T21:46:19.852ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2652024-12-20T21:46:19.852ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2662024-12-20T21:46:19.853ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2672024-12-20T21:46:19.853ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2682024-12-20T21:46:19.902ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2692024-12-20T21:46:19.902ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2702024-12-20T21:46:19.902ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2712024-12-20T21:46:19.902ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2722024-12-20T21:46:19.903ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2732024-12-20T21:46:19.908ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2742024-12-20T21:46:19.909ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2752024-12-20T21:46:19.909ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2762024-12-20T21:46:19.909ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772024-12-20T21:46:19.911ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2782024-12-20T21:46:19.911ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2792024-12-20T21:46:19.911ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2802024-12-20T21:46:19.911ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2812024-12-20T21:46:19.913ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2822024-12-20T21:46:19.923ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2832024-12-20T21:46:19.924ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2842024-12-20T21:46:19.924ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2852024-12-20T21:46:19.924ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2862024-12-20T21:46:19.963ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2872024-12-20T21:46:19.963ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2882024-12-20T21:46:19.964ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2892024-12-20T21:46:19.998ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2902024-12-20T21:46:19.998ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2912024-12-20T21:46:19.999ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2922024-12-20T21:46:20.004ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2932024-12-20T21:46:20.006ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2942024-12-20T21:46:20.006ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2952024-12-20T21:46:20.006ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
2962024-12-20T21:46:20.006ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2972024-12-20T21:46:20.012ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2982024-12-20T21:46:20.012ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2992024-12-20T21:46:20.013ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3002024-12-20T21:46:20.013ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3012024-12-20T21:46:20.015ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3022024-12-20T21:46:20.016ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3032024-12-20T21:46:20.016ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3042024-12-20T21:46:20.016ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3052024-12-20T21:46:20.077ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3062024-12-20T21:46:20.083ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3072024-12-20T21:46:20.084ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3082024-12-20T21:46:20.084ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3092024-12-20T21:46:20.084ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3102024-12-20T21:46:20.133ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3112024-12-20T21:46:20.133ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3122024-12-20T21:46:20.133ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3132024-12-20T21:46:20.133ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3142024-12-20T21:46:20.134ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152024-12-20T21:46:20.141ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3162024-12-20T21:46:20.142ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3172024-12-20T21:46:20.143ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3182024-12-20T21:46:20.143ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192024-12-20T21:46:20.143ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3202024-12-20T21:46:20.143ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3212024-12-20T21:46:20.143ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3222024-12-20T21:46:20.143ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3232024-12-20T21:46:20.145ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3242024-12-20T21:46:20.151ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3252024-12-20T21:46:20.153ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3262024-12-20T21:46:20.153ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3272024-12-20T21:46:20.153ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3282024-12-20T21:46:20.317ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3292024-12-20T21:46:20.321ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3302024-12-20T21:46:20.321ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3312024-12-20T21:46:20.321ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3322024-12-20T21:46:20.321ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332024-12-20T21:46:20.328ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3342024-12-20T21:46:20.328ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3352024-12-20T21:46:20.328ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3362024-12-20T21:46:20.328ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372024-12-20T21:46:20.335ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3382024-12-20T21:46:20.336ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3392024-12-20T21:46:20.336ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3402024-12-20T21:46:20.336ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3412024-12-20T21:46:20.342ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3422024-12-20T21:46:20.351ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3432024-12-20T21:46:20.352ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3442024-12-20T21:46:20.352ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3452024-12-20T21:46:20.352ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3462024-12-20T21:46:20.366ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPC firewall rules
3472024-12-20T21:46:20.373ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3482024-12-20T21:46:20.373ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3492024-12-20T21:46:20.373ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3502024-12-20T21:46:20.373ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3512024-12-20T21:46:20.373ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3522024-12-20T21:46:20.373ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3532024-12-20T21:46:20.374ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3542024-12-20T21:46:20.374ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3552024-12-20T21:46:20.375ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3562024-12-20T21:46:20.385ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3572024-12-20T21:46:20.385ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3582024-12-20T21:46:20.385ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3592024-12-20T21:46:20.385ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3602024-12-20T21:46:20.385ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3612024-12-20T21:46:20.386ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3622024-12-20T21:46:20.386ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3632024-12-20T21:46:20.386ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3642024-12-20T21:46:20.386ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3652024-12-20T21:46:20.386ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3662024-12-20T21:46:20.386ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3672024-12-20T21:46:20.386ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3682024-12-20T21:46:20.386ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3692024-12-20T21:46:20.386ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3702024-12-20T21:46:20.391ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3712024-12-20T21:46:20.391ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
3722024-12-20T21:46:20.391ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3732024-12-20T21:46:20.395ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3742024-12-20T21:46:20.395ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3752024-12-20T21:46:20.395ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3762024-12-20T21:46:20.395ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3772024-12-20T21:46:20.395ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3782024-12-20T21:46:20.395ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3792024-12-20T21:46:20.395ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3802024-12-20T21:46:20.395ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3812024-12-20T21:46:20.397ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3822024-12-20T21:46:20.397ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3832024-12-20T21:46:20.397ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3842024-12-20T21:46:20.397ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3852024-12-20T21:46:20.399ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3862024-12-20T21:46:20.407ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3872024-12-20T21:46:20.421ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3882024-12-20T21:46:20.421ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3892024-12-20T21:46:20.421ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3902024-12-20T21:46:20.463ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3912024-12-20T21:46:20.463ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3922024-12-20T21:46:20.463ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3932024-12-20T21:46:20.463ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3942024-12-20T21:46:20.464ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952024-12-20T21:46:20.465ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3962024-12-20T21:46:20.466ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3972024-12-20T21:46:20.467ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
3982024-12-20T21:46:20.468ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992024-12-20T21:46:20.468ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4002024-12-20T21:46:20.470ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4012024-12-20T21:46:20.470ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4022024-12-20T21:46:20.470ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4032024-12-20T21:46:20.472ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4042024-12-20T21:46:20.479ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4052024-12-20T21:46:20.479ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4062024-12-20T21:46:20.479ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4072024-12-20T21:46:20.498ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4082024-12-20T21:46:20.601ZINFOtest_mgs_metrics (DataLoader): created built-in VPC firewall rules
4092024-12-20T21:46:20.601ZDEBGtest_mgs_metrics (DataLoader): attempting to create built-in VPC Subnets
4102024-12-20T21:46:20.601ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4112024-12-20T21:46:20.601ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4122024-12-20T21:46:20.601ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4132024-12-20T21:46:20.617ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4142024-12-20T21:46:20.617ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4152024-12-20T21:46:20.618ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4162024-12-20T21:46:20.619ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4172024-12-20T21:46:20.619ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4182024-12-20T21:46:20.620ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4192024-12-20T21:46:20.621ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4202024-12-20T21:46:20.621ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4212024-12-20T21:46:20.622ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4222024-12-20T21:46:20.622ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4232024-12-20T21:46:20.622ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4242024-12-20T21:46:20.623ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4252024-12-20T21:46:20.624ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4262024-12-20T21:46:20.624ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4272024-12-20T21:46:20.624ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4282024-12-20T21:46:20.625ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4292024-12-20T21:46:20.625ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4302024-12-20T21:46:20.625ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4312024-12-20T21:46:20.626ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4322024-12-20T21:46:20.627ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4332024-12-20T21:46:20.637ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4342024-12-20T21:46:20.637ZTRACtest_mgs_metrics (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
4352024-12-20T21:46:20.637ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4362024-12-20T21:46:20.637ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4372024-12-20T21:46:20.637ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4382024-12-20T21:46:20.682ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4392024-12-20T21:46:20.682ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4402024-12-20T21:46:20.683ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4412024-12-20T21:46:20.683ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4422024-12-20T21:46:20.683ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4432024-12-20T21:46:20.685ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4442024-12-20T21:46:20.685ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4452024-12-20T21:46:20.685ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4462024-12-20T21:46:20.685ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4472024-12-20T21:46:20.687ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4482024-12-20T21:46:20.687ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4492024-12-20T21:46:20.687ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4502024-12-20T21:46:20.689ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4512024-12-20T21:46:20.690ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4522024-12-20T21:46:20.696ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4532024-12-20T21:46:20.708ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4542024-12-20T21:46:20.709ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4552024-12-20T21:46:20.709ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4562024-12-20T21:46:20.742ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4572024-12-20T21:46:20.744ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4582024-12-20T21:46:20.746ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4592024-12-20T21:46:20.752ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4602024-12-20T21:46:20.753ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4612024-12-20T21:46:20.753ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4622024-12-20T21:46:20.753ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4632024-12-20T21:46:20.753ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4642024-12-20T21:46:20.757ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4652024-12-20T21:46:20.758ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4662024-12-20T21:46:20.758ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4672024-12-20T21:46:20.758ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4682024-12-20T21:46:20.759ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4692024-12-20T21:46:20.760ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4702024-12-20T21:46:20.760ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4712024-12-20T21:46:20.761ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4722024-12-20T21:46:20.763ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4732024-12-20T21:46:20.770ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
4742024-12-20T21:46:20.770ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4752024-12-20T21:46:20.770ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4762024-12-20T21:46:20.770ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4772024-12-20T21:46:20.790ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4782024-12-20T21:46:20.790ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4792024-12-20T21:46:20.790ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4802024-12-20T21:46:20.792ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4812024-12-20T21:46:20.794ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4822024-12-20T21:46:20.797ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4832024-12-20T21:46:20.799ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4842024-12-20T21:46:20.799ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4852024-12-20T21:46:20.799ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4862024-12-20T21:46:20.799ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4872024-12-20T21:46:20.799ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4882024-12-20T21:46:20.799ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4892024-12-20T21:46:20.799ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4902024-12-20T21:46:20.799ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4912024-12-20T21:46:20.799ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4922024-12-20T21:46:20.799ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4932024-12-20T21:46:20.799ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4942024-12-20T21:46:20.806ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4952024-12-20T21:46:20.826ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4962024-12-20T21:46:20.826ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
4972024-12-20T21:46:20.828ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4982024-12-20T21:46:20.858ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4992024-12-20T21:46:20.858ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5002024-12-20T21:46:20.859ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5012024-12-20T21:46:20.867ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5022024-12-20T21:46:20.867ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5032024-12-20T21:46:20.867ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5042024-12-20T21:46:20.871ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5052024-12-20T21:46:20.871ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5062024-12-20T21:46:20.877ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5072024-12-20T21:46:20.878ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5082024-12-20T21:46:20.878ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5092024-12-20T21:46:20.878ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5102024-12-20T21:46:20.880ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5112024-12-20T21:46:20.881ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5122024-12-20T21:46:20.881ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5132024-12-20T21:46:20.881ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5142024-12-20T21:46:20.883ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5152024-12-20T21:46:20.892ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5162024-12-20T21:46:20.892ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5172024-12-20T21:46:20.892ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5182024-12-20T21:46:20.893ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5192024-12-20T21:46:20.905ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5202024-12-20T21:46:20.905ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5212024-12-20T21:46:20.907ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5222024-12-20T21:46:20.909ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
5232024-12-20T21:46:20.909ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5242024-12-20T21:46:20.909ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5252024-12-20T21:46:20.909ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5262024-12-20T21:46:20.909ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5272024-12-20T21:46:20.911ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5282024-12-20T21:46:20.912ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5292024-12-20T21:46:20.912ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5302024-12-20T21:46:20.912ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5312024-12-20T21:46:20.913ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5322024-12-20T21:46:20.913ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5332024-12-20T21:46:20.913ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5342024-12-20T21:46:20.914ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5352024-12-20T21:46:20.916ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5362024-12-20T21:46:20.925ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5372024-12-20T21:46:20.943ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5382024-12-20T21:46:20.943ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5392024-12-20T21:46:20.944ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5402024-12-20T21:46:20.968ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5412024-12-20T21:46:20.968ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5422024-12-20T21:46:20.973ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5432024-12-20T21:46:20.986ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5442024-12-20T21:46:20.986ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5452024-12-20T21:46:20.986ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5462024-12-20T21:46:20.986ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5472024-12-20T21:46:20.987ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5482024-12-20T21:46:20.991ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5492024-12-20T21:46:20.991ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5502024-12-20T21:46:20.993ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5512024-12-20T21:46:20.993ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5522024-12-20T21:46:20.994ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5532024-12-20T21:46:20.994ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5542024-12-20T21:46:20.994ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5552024-12-20T21:46:20.995ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5562024-12-20T21:46:21.006ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5572024-12-20T21:46:21.014ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5582024-12-20T21:46:21.014ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5592024-12-20T21:46:21.016ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5602024-12-20T21:46:21.016ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5612024-12-20T21:46:21.035ZINFOtest_mgs_metrics (DataLoader): created built-in services vpc subnets
5622024-12-20T21:46:21.035ZINFOtest_mgs_metrics (DataLoader): created built-in services vpc
5632024-12-20T21:46:21.035ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5642024-12-20T21:46:21.035ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5652024-12-20T21:46:21.038ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5662024-12-20T21:46:21.038ZDEBGtest_mgs_metrics (DataLoader): attempting to create silo users
5672024-12-20T21:46:21.038ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5682024-12-20T21:46:21.038ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5692024-12-20T21:46:21.039ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5702024-12-20T21:46:21.076ZINFOtest_mgs_metrics (DataLoader): created 2 silo users
5712024-12-20T21:46:21.076ZDEBGtest_mgs_metrics (DataLoader): attempting to create silo user role assignments
5722024-12-20T21:46:21.076ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5732024-12-20T21:46:21.076ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5742024-12-20T21:46:21.076ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5752024-12-20T21:46:21.083ZINFOtest_mgs_metrics (DataLoader): created 2 silo user role assignments
5762024-12-20T21:46:21.083ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5772024-12-20T21:46:21.084ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5782024-12-20T21:46:21.084ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5792024-12-20T21:46:21.094ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5802024-12-20T21:46:21.094ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5812024-12-20T21:46:21.095ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5822024-12-20T21:46:21.110ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5832024-12-20T21:46:21.113ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5842024-12-20T21:46:21.113ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5852024-12-20T21:46:21.113ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5862024-12-20T21:46:21.113ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5872024-12-20T21:46:21.113ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5882024-12-20T21:46:21.114ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5892024-12-20T21:46:21.114ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5902024-12-20T21:46:21.114ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5912024-12-20T21:46:21.115ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5922024-12-20T21:46:21.148ZTRACtest_mgs_metrics (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5932024-12-20T21:46:21.149ZTRACtest_mgs_metrics (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5942024-12-20T21:46:21.149ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5952024-12-20T21:46:21.149ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5962024-12-20T21:46:21.149ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5972024-12-20T21:46:21.152ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
5982024-12-20T21:46:21.154ZDEBGtest_mgs_metrics (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5992024-12-20T21:46:21.155ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6002024-12-20T21:46:21.155ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
6012024-12-20T21:46:21.155ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6022024-12-20T21:46:21.308ZTRACtest_mgs_metrics (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6032024-12-20T21:46:21.309ZDEBGtest_mgs_metrics (DataLoader): roles roles = RoleSet { roles: {} }
6042024-12-20T21:46:21.309ZDEBGtest_mgs_metrics (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6052024-12-20T21:46:21.316ZINFOtest_mgs_metrics (nexus): populate complete
6062024-12-20T21:46:21.316ZINFOtest_mgs_metrics (nexus): activating background tasks
6072024-12-20T21:46:21.318ZDEBGtest_mgs_metrics (init_with_steps): Running step start_sled1
6082024-12-20T21:46:21.318ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6092024-12-20T21:46:21.356ZINFOtest_mgs_metrics (SledAgent): created simulated sled agent server = b6d65341-167c-41df-9b5c-41cded99c229 sim_mode = Explicit sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6102024-12-20T21:46:21.398ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /artifacts sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6112024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = DELETE path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6122024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6132024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6142024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6152024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6162024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6172024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /bootstore/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6182024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /cockroachdb sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6192024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6202024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6212024-12-20T21:46:21.399ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /disks/{disk_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6222024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /disks/{disk_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6232024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /eip-gateways sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6242024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /inventory sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6252024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6262024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6272024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6282024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6292024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /omicron-zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6302024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /sled-identifiers sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6312024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /sled-role sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6322024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /sleds sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6332024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /support/dladm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6342024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /support/ipadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6352024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /support/zoneadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6362024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /support-bundles/{zpool_id}/{dataset_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6372024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = DELETE path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6382024-12-20T21:46:21.400ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6392024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6402024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6412024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6422024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6432024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6442024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6452024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /switch-ports sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6462024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /timesync sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6472024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /update sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6482024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = DELETE path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6492024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6502024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6512024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = DELETE path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6522024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6532024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6542024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6552024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6562024-12-20T21:46:21.401ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /vmms/{propolis_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6572024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6582024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6592024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6602024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6612024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6622024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6632024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6642024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6652024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /zones/bundle-cleanup sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6662024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6672024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = PUT path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6682024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /zones/bundle-cleanup/utilization sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6692024-12-20T21:46:21.402ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /zones/bundles sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6702024-12-20T21:46:21.403ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6712024-12-20T21:46:21.403ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = POST path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6722024-12-20T21:46:21.403ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6732024-12-20T21:46:21.403ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6742024-12-20T21:46:21.404ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34047 method = GET path = /zpools sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
6752024-12-20T21:46:21.404ZINFOtest_mgs_metrics (dropshot): listening local_addr = [::1]:34047 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6762024-12-20T21:46:21.404ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:34047 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6772024-12-20T21:46:21.404ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
6782024-12-20T21:46:21.404ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6792024-12-20T21:46:21.404ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6802024-12-20T21:46:21.404ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6812024-12-20T21:46:21.404ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6822024-12-20T21:46:21.404ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
6832024-12-20T21:46:21.404ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
6842024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
6852024-12-20T21:46:21.405ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config nor servers
6862024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 41.791\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6872024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
6882024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6892024-12-20T21:46:21.405ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
6902024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6912024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6922024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
6932024-12-20T21:46:21.405ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
6942024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
6952024-12-20T21:46:21.405ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config nor servers
6962024-12-20T21:46:21.405ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 25.89\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6972024-12-20T21:46:21.406ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
6982024-12-20T21:46:21.406ZINFOtest_mgs_metrics (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-20 21:36:21.401460410 UTC
6992024-12-20T21:46:21.406ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
7002024-12-20T21:46:21.406ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
7012024-12-20T21:46:21.406ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
7022024-12-20T21:46:21.407ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
7032024-12-20T21:46:21.407ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
7042024-12-20T21:46:21.407ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7052024-12-20T21:46:21.408ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
7062024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7072024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7082024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
7092024-12-20T21:46:21.408ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
7102024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7112024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7122024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
7132024-12-20T21:46:21.408ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
7142024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7152024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7162024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
7172024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
7182024-12-20T21:46:21.408ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
7192024-12-20T21:46:21.408ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7202024-12-20T21:46:21.408ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
7212024-12-20T21:46:21.408ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7222024-12-20T21:46:21.409ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7232024-12-20T21:46:21.409ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
7242024-12-20T21:46:21.409ZWARNtest_mgs_metrics (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = no blueprint
7252024-12-20T21:46:21.409ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 25.64\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7262024-12-20T21:46:21.409ZDEBGtest_mgs_metrics (omicron_sled_agent::sim::Server): contacting server nexus sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7272024-12-20T21:46:21.409ZDEBGtest_mgs_metrics (NexusClient): client request body = Some(Body) method = POST sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = http://127.0.0.1:38423/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
7282024-12-20T21:46:21.409ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54972
7292024-12-20T21:46:21.409ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
7302024-12-20T21:46:21.411ZWARNtest_mgs_metrics (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector reason = no blueprint
7312024-12-20T21:46:21.411ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 80.301\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7322024-12-20T21:46:21.411ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
7332024-12-20T21:46:21.411ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
7342024-12-20T21:46:21.411ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7352024-12-20T21:46:21.411ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
7362024-12-20T21:46:21.411ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7372024-12-20T21:46:21.411ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7382024-12-20T21:46:21.411ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
7392024-12-20T21:46:21.411ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 29.83\u{b5}s iteration = 1 status = {"error":"task disabled"}
7402024-12-20T21:46:21.411ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
7412024-12-20T21:46:21.412ZINFOtest_mgs_metrics (BackgroundTasks): Decommissioned Disk Cleaner disabled background_task = decommissioned_disk_cleaner
7422024-12-20T21:46:21.412ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 25.231\u{b5}s iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7432024-12-20T21:46:21.412ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
7442024-12-20T21:46:21.412ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
7452024-12-20T21:46:21.412ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7462024-12-20T21:46:21.412ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
7472024-12-20T21:46:21.412ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7482024-12-20T21:46:21.413ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7492024-12-20T21:46:21.413ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
7502024-12-20T21:46:21.413ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
7512024-12-20T21:46:21.413ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7522024-12-20T21:46:21.413ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
7532024-12-20T21:46:21.413ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7542024-12-20T21:46:21.413ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7552024-12-20T21:46:21.413ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
7562024-12-20T21:46:21.414ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
7572024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7582024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7592024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
7602024-12-20T21:46:21.414ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
7612024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7622024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7632024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
7642024-12-20T21:46:21.414ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
7652024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7662024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7672024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
7682024-12-20T21:46:21.414ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
7692024-12-20T21:46:21.414ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7702024-12-20T21:46:21.414ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
7712024-12-20T21:46:21.414ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7722024-12-20T21:46:21.415ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7732024-12-20T21:46:21.415ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
7742024-12-20T21:46:21.415ZINFOtest_mgs_metrics (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
7752024-12-20T21:46:21.415ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_updater elapsed = 102.062\u{b5}s iteration = 1 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7762024-12-20T21:46:21.415ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
7772024-12-20T21:46:21.415ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
7782024-12-20T21:46:21.415ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7792024-12-20T21:46:21.416ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7802024-12-20T21:46:21.417ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
7812024-12-20T21:46:21.417ZDEBGtest_mgs_metrics (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7822024-12-20T21:46:21.417ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
7832024-12-20T21:46:21.417ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7842024-12-20T21:46:21.417ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7852024-12-20T21:46:21.418ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
7862024-12-20T21:46:21.418ZINFOtest_mgs_metrics (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
7872024-12-20T21:46:21.418ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
7882024-12-20T21:46:21.418ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7892024-12-20T21:46:21.418ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
7902024-12-20T21:46:21.418ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7912024-12-20T21:46:21.418ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7922024-12-20T21:46:21.418ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
7932024-12-20T21:46:21.418ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
7942024-12-20T21:46:21.418ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7952024-12-20T21:46:21.418ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7962024-12-20T21:46:21.418ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
7972024-12-20T21:46:21.421ZDEBGtest_mgs_metrics (SagaRecoverer): listing candidate sagas for recovery
7982024-12-20T21:46:21.421ZTRACtest_mgs_metrics (SagaRecoverer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database
7992024-12-20T21:46:21.421ZDEBGtest_mgs_metrics (SagaRecoverer): roles roles = RoleSet { roles: {} }
8002024-12-20T21:46:21.421ZDEBGtest_mgs_metrics (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
8012024-12-20T21:46:21.421ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
8022024-12-20T21:46:21.421ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
8032024-12-20T21:46:21.421ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
8042024-12-20T21:46:21.421ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
8052024-12-20T21:46:21.423ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
8062024-12-20T21:46:21.423ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
8072024-12-20T21:46:21.423ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8082024-12-20T21:46:21.423ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8092024-12-20T21:46:21.424ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
8102024-12-20T21:46:21.424ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database
8112024-12-20T21:46:21.424ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
8122024-12-20T21:46:21.424ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8132024-12-20T21:46:21.424ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
8142024-12-20T21:46:21.425ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
8152024-12-20T21:46:21.426ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8162024-12-20T21:46:21.426ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8172024-12-20T21:46:21.426ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
8182024-12-20T21:46:21.426ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database
8192024-12-20T21:46:21.426ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
8202024-12-20T21:46:21.427ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8212024-12-20T21:46:21.427ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
8222024-12-20T21:46:21.427ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8232024-12-20T21:46:21.427ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
8242024-12-20T21:46:21.427ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8252024-12-20T21:46:21.427ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8262024-12-20T21:46:21.427ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54972 req_id = e417ec78-3a01-44be-a23c-76e53b811e8e uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8272024-12-20T21:46:21.427ZINFOtest_mgs_metrics (nexus): registered sled agent sled_uuid = b6d65341-167c-41df-9b5c-41cded99c229
8282024-12-20T21:46:21.427ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: 75 5d 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 u]..........._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:54965 req_id = bde298b2-4e2f-4c2b-9344-998d2aa7892c
8292024-12-20T21:46:21.427ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 30045,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54965 req_id = bde298b2-4e2f-4c2b-9344-998d2aa7892c
8302024-12-20T21:46:21.427ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30045, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_nameservice._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 12, 95, 110, 97, 109, 101, 115, 101, 114, 118, 105, 99, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_nameservice._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54965 req_id = bde298b2-4e2f-4c2b-9344-998d2aa7892c
8312024-12-20T21:46:21.428ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: af 52 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 .R..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63010 req_id = 014a6da8-1584-4378-9e1f-3276c72b7b25
8322024-12-20T21:46:21.428ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44882,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63010 req_id = 014a6da8-1584-4378-9e1f-3276c72b7b25
8332024-12-20T21:46:21.428ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44882, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_external-dns._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 13, 95, 101, 120, 116, 101, 114, 110, 97, 108, 45, 100, 110, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_external-dns._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63010 req_id = 014a6da8-1584-4378-9e1f-3276c72b7b25
8342024-12-20T21:46:21.428ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
8352024-12-20T21:46:21.428ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8362024-12-20T21:46:21.428ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
8372024-12-20T21:46:21.428ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8382024-12-20T21:46:21.428ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8392024-12-20T21:46:21.428ZWARNtest_mgs_metrics (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_internal dns_group = internal error = no record found for Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8402024-12-20T21:46:21.428ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 25.968066ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_nameservice._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8412024-12-20T21:46:21.428ZWARNtest_mgs_metrics (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_external dns_group = external error = no record found for Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8422024-12-20T21:46:21.429ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 24.996411ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_external-dns._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8432024-12-20T21:46:21.460ZINFOtest_mgs_metrics (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-20 21:36:21.401460410 UTC pruned = {}
8442024-12-20T21:46:21.460ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 59.339551ms iteration = 1 status = {"expiration":"2024-12-20T21:36:21.401460410Z","pruned":[]}
8452024-12-20T21:46:21.500ZTRACtest_mgs_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8462024-12-20T21:46:21.502ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: e6 70 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .p..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:58012 req_id = ad632f07-5364-46c0-bca0-2940a519a14e
8472024-12-20T21:46:21.503ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58992,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58012 req_id = ad632f07-5364-46c0-bca0-2940a519a14e
8482024-12-20T21:46:21.503ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 58992, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58012 req_id = ad632f07-5364-46c0-bca0-2940a519a14e
8492024-12-20T21:46:21.503ZERROtest_mgs_metrics (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8502024-12-20T21:46:21.504ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 100.363139ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8512024-12-20T21:46:21.505ZTRACtest_mgs_metrics (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8522024-12-20T21:46:21.505ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 97 d0 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:51835 req_id = 29a1ccbd-315f-454c-aed4-5d91940d5052
8532024-12-20T21:46:21.505ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38864,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51835 req_id = 29a1ccbd-315f-454c-aed4-5d91940d5052
8542024-12-20T21:46:21.505ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38864, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51835 req_id = 29a1ccbd-315f-454c-aed4-5d91940d5052
8552024-12-20T21:46:21.505ZERROtest_mgs_metrics (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8562024-12-20T21:46:21.505ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 101.561828ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8572024-12-20T21:46:21.518ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
8582024-12-20T21:46:21.521ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8592024-12-20T21:46:21.521ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8602024-12-20T21:46:21.574ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 167.229592ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8612024-12-20T21:46:21.579ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8622024-12-20T21:46:21.584ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8632024-12-20T21:46:21.584ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
8642024-12-20T21:46:21.584ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8652024-12-20T21:46:21.585ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8662024-12-20T21:46:21.587ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8672024-12-20T21:46:21.589ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8682024-12-20T21:46:21.591ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
8692024-12-20T21:46:21.591ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8702024-12-20T21:46:21.591ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8712024-12-20T21:46:21.603ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8722024-12-20T21:46:21.607ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8732024-12-20T21:46:21.607ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
8742024-12-20T21:46:21.607ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8752024-12-20T21:46:21.615ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8762024-12-20T21:46:21.784ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 365.640212ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
8772024-12-20T21:46:21.784ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8782024-12-20T21:46:21.789ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8792024-12-20T21:46:21.789ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
8802024-12-20T21:46:21.789ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8812024-12-20T21:46:21.789ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8822024-12-20T21:46:21.806ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8832024-12-20T21:46:21.814ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8842024-12-20T21:46:21.879ZDEBGtest_mgs_metrics (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
8852024-12-20T21:46:21.879ZINFOtest_mgs_metrics (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
8862024-12-20T21:46:21.879ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 461.794343ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
8872024-12-20T21:46:21.931ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
8882024-12-20T21:46:21.933ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8892024-12-20T21:46:21.934ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8902024-12-20T21:46:21.939ZINFOtest_mgs_metrics (SagaRecoverer): listed in-progress sagas count = 0
8912024-12-20T21:46:21.939ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 520.130479ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
8922024-12-20T21:46:21.941ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8932024-12-20T21:46:21.953ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8942024-12-20T21:46:21.953ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
8952024-12-20T21:46:21.953ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8962024-12-20T21:46:21.953ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8972024-12-20T21:46:21.954ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8982024-12-20T21:46:21.960ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8992024-12-20T21:46:21.960ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
9002024-12-20T21:46:21.960ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9012024-12-20T21:46:21.960ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9022024-12-20T21:46:21.960ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9032024-12-20T21:46:21.961ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9042024-12-20T21:46:21.966ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9052024-12-20T21:46:21.970ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9062024-12-20T21:46:21.970ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
9072024-12-20T21:46:21.970ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9082024-12-20T21:46:21.971ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9092024-12-20T21:46:21.991ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 571.549113ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
9102024-12-20T21:46:21.997ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9112024-12-20T21:46:21.997ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9122024-12-20T21:46:21.998ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9132024-12-20T21:46:22.025ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
9142024-12-20T21:46:22.027ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9152024-12-20T21:46:22.027ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9162024-12-20T21:46:22.027ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
9172024-12-20T21:46:22.027ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9182024-12-20T21:46:22.028ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9192024-12-20T21:46:22.039ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 618.033529ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
9202024-12-20T21:46:22.040ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
9212024-12-20T21:46:22.040ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9222024-12-20T21:46:22.041ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9232024-12-20T21:46:22.041ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9242024-12-20T21:46:22.041ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9252024-12-20T21:46:22.041ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 634.034646ms iteration = 1 status = {}
9262024-12-20T21:46:22.056ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9272024-12-20T21:46:22.068ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9282024-12-20T21:46:22.069ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9292024-12-20T21:46:22.069ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9302024-12-20T21:46:22.069ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9312024-12-20T21:46:22.091ZWARNtest_mgs_metrics (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0
9322024-12-20T21:46:22.091ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 692.091356ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
9332024-12-20T21:46:22.092ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
9342024-12-20T21:46:22.092ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9352024-12-20T21:46:22.093ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9362024-12-20T21:46:22.124ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9372024-12-20T21:46:22.131ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9382024-12-20T21:46:22.133ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
9392024-12-20T21:46:22.133ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
9402024-12-20T21:46:22.133ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9412024-12-20T21:46:22.133ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9422024-12-20T21:46:22.133ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9432024-12-20T21:46:22.133ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9442024-12-20T21:46:22.138ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9452024-12-20T21:46:22.138ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9462024-12-20T21:46:22.138ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9472024-12-20T21:46:22.138ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9482024-12-20T21:46:22.138ZWARNtest_mgs_metrics (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set
9492024-12-20T21:46:22.138ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 734.731039ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
9502024-12-20T21:46:22.139ZTRACtest_mgs_metrics (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = instance failed total_found = 0
9512024-12-20T21:46:22.139ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
9522024-12-20T21:46:22.139ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9532024-12-20T21:46:22.139ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9542024-12-20T21:46:22.176ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker
9552024-12-20T21:46:22.176ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 769.358495ms iteration = 1 status = {"error":"inventory collection is None"}
9562024-12-20T21:46:22.177ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
9572024-12-20T21:46:22.179ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
9582024-12-20T21:46:22.179ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 51 91 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 Q............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59416 req_id = 1cac3e4c-f141-490f-9293-b1373697b34a
9592024-12-20T21:46:22.179ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20881,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59416 req_id = 1cac3e4c-f141-490f-9293-b1373697b34a
9602024-12-20T21:46:22.179ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 20881, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59416 req_id = 1cac3e4c-f141-490f-9293-b1373697b34a
9612024-12-20T21:46:22.179ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
9622024-12-20T21:46:22.179ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 771.996367ms iteration = 1 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
9632024-12-20T21:46:22.189ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
9642024-12-20T21:46:22.200ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9652024-12-20T21:46:22.200ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9662024-12-20T21:46:22.200ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9672024-12-20T21:46:22.200ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 779.733551ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9682024-12-20T21:46:22.208ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 797.499236ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9692024-12-20T21:46:22.236ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9702024-12-20T21:46:22.236ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9712024-12-20T21:46:22.236ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9722024-12-20T21:46:22.238ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9732024-12-20T21:46:22.246ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9742024-12-20T21:46:22.249ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
9752024-12-20T21:46:22.249ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
9762024-12-20T21:46:22.249ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
9772024-12-20T21:46:22.249ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
9782024-12-20T21:46:22.249ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9792024-12-20T21:46:22.250ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9802024-12-20T21:46:22.251ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9812024-12-20T21:46:22.252ZWARNtest_mgs_metrics (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0
9822024-12-20T21:46:22.252ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 850.873192ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
9832024-12-20T21:46:22.264ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
9842024-12-20T21:46:22.264ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9852024-12-20T21:46:22.264ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9862024-12-20T21:46:22.265ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9872024-12-20T21:46:22.265ZINFOtest_mgs_metrics (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9882024-12-20T21:46:22.265ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 852.968336ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
9892024-12-20T21:46:22.265ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_replacement elapsed = 855.241121ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9902024-12-20T21:46:22.271ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
9912024-12-20T21:46:22.272ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9922024-12-20T21:46:22.273ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9932024-12-20T21:46:22.274ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9942024-12-20T21:46:22.274ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9952024-12-20T21:46:22.274ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9962024-12-20T21:46:22.275ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9972024-12-20T21:46:22.286ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9982024-12-20T21:46:22.298ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
9992024-12-20T21:46:22.299ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10002024-12-20T21:46:22.300ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10012024-12-20T21:46:22.301ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10022024-12-20T21:46:22.330ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10032024-12-20T21:46:22.333ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10042024-12-20T21:46:22.333ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10052024-12-20T21:46:22.334ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10062024-12-20T21:46:22.336ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10072024-12-20T21:46:22.336ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10082024-12-20T21:46:22.338ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10092024-12-20T21:46:22.338ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10102024-12-20T21:46:22.338ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10112024-12-20T21:46:22.341ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
10122024-12-20T21:46:22.341ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
10132024-12-20T21:46:22.341ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10142024-12-20T21:46:22.341ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10152024-12-20T21:46:22.343ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10162024-12-20T21:46:22.344ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
10172024-12-20T21:46:22.346ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10182024-12-20T21:46:22.346ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10192024-12-20T21:46:22.346ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10202024-12-20T21:46:22.347ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10212024-12-20T21:46:22.347ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10222024-12-20T21:46:22.347ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10232024-12-20T21:46:22.353ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10242024-12-20T21:46:22.365ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10252024-12-20T21:46:22.377ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
10262024-12-20T21:46:22.382ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
10272024-12-20T21:46:22.382ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10282024-12-20T21:46:22.382ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10292024-12-20T21:46:22.383ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10302024-12-20T21:46:22.386ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 966.147489ms iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
10312024-12-20T21:46:22.387ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
10322024-12-20T21:46:22.387ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
10332024-12-20T21:46:22.388ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10342024-12-20T21:46:22.388ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
10352024-12-20T21:46:22.388ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10362024-12-20T21:46:22.389ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
10372024-12-20T21:46:22.391ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 968082 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54972 req_id = e417ec78-3a01-44be-a23c-76e53b811e8e response_code = 204 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
10382024-12-20T21:46:22.395ZDEBGtest_mgs_metrics (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:38423/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229", status: 204, headers: {"x-request-id": "e417ec78-3a01-44be-a23c-76e53b811e8e", "date": "Fri, 20 Dec 2024 21:46:21 GMT"} }) sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10392024-12-20T21:46:22.395ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10402024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10412024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10422024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10432024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10442024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10452024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10462024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10472024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42603 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10482024-12-20T21:46:22.397ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:42603 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10492024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:42603 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10502024-12-20T21:46:22.397ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:42603 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10512024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10522024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10532024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10542024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10552024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10562024-12-20T21:46:22.397ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10572024-12-20T21:46:22.398ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10582024-12-20T21:46:22.398ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10592024-12-20T21:46:22.398ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45775 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10602024-12-20T21:46:22.399ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45775 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10612024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45775 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10622024-12-20T21:46:22.399ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45775 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10632024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10642024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10652024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10662024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10672024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10682024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10692024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10702024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10712024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:42265 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10722024-12-20T21:46:22.399ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:42265 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10732024-12-20T21:46:22.399ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:42265 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10742024-12-20T21:46:22.400ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:42265 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10752024-12-20T21:46:22.400ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10762024-12-20T21:46:22.406ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10772024-12-20T21:46:22.406ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10782024-12-20T21:46:22.406ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10792024-12-20T21:46:22.406ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10802024-12-20T21:46:22.407ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10812024-12-20T21:46:22.407ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10822024-12-20T21:46:22.407ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10832024-12-20T21:46:22.407ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46575 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10842024-12-20T21:46:22.407ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:46575 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10852024-12-20T21:46:22.407ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:46575 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10862024-12-20T21:46:22.407ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:46575 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10872024-12-20T21:46:22.408ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10882024-12-20T21:46:22.408ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10892024-12-20T21:46:22.408ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10902024-12-20T21:46:22.408ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10912024-12-20T21:46:22.408ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10922024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10932024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10942024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10952024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:40459 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10962024-12-20T21:46:22.409ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:40459 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10972024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:40459 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10982024-12-20T21:46:22.409ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:40459 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10992024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11002024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11012024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11022024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11032024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11042024-12-20T21:46:22.409ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11052024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11062024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11072024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37995 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11082024-12-20T21:46:22.410ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:37995 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11092024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:37995 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11102024-12-20T21:46:22.410ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:37995 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11112024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11122024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11132024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11142024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11152024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11162024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11172024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11182024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11192024-12-20T21:46:22.410ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33387 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11202024-12-20T21:46:22.411ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:33387 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11212024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:33387 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11222024-12-20T21:46:22.411ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:33387 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11232024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11242024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11252024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11262024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11272024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11282024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11292024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11302024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11312024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:46349 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11322024-12-20T21:46:22.411ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:46349 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11332024-12-20T21:46:22.411ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:46349 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11342024-12-20T21:46:22.411ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:46349 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11352024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11362024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11372024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11382024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11392024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11402024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11412024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11422024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11432024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:37049 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11442024-12-20T21:46:22.412ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:37049 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11452024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:37049 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11462024-12-20T21:46:22.412ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:37049 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11472024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11482024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11492024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11502024-12-20T21:46:22.412ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11512024-12-20T21:46:22.413ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11522024-12-20T21:46:22.413ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11532024-12-20T21:46:22.413ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11542024-12-20T21:46:22.413ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11552024-12-20T21:46:22.413ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35061 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
11562024-12-20T21:46:22.413ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:35061 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11572024-12-20T21:46:22.413ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:35061 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11582024-12-20T21:46:22.413ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:35061 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11592024-12-20T21:46:22.413ZDEBGtest_mgs_metrics (init_with_steps): Running step start_sled2
11602024-12-20T21:46:22.413ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11612024-12-20T21:46:22.460ZINFOtest_mgs_metrics (SledAgent): created simulated sled agent server = 039be560-54cc-49e3-88df-1a29dadbf913 sim_mode = Explicit sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
11622024-12-20T21:46:22.525ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /artifacts sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11632024-12-20T21:46:22.526ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = DELETE path = /artifacts/{sha256} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11642024-12-20T21:46:22.526ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /artifacts/{sha256} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11652024-12-20T21:46:22.526ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11662024-12-20T21:46:22.526ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11672024-12-20T21:46:22.527ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11682024-12-20T21:46:22.527ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11692024-12-20T21:46:22.527ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /bootstore/status sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11702024-12-20T21:46:22.527ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /cockroachdb sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11712024-12-20T21:46:22.527ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /datasets sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11722024-12-20T21:46:22.527ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /datasets sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11732024-12-20T21:46:22.527ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /disks/{disk_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11742024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /disks/{disk_id}/poke sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11752024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /eip-gateways sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11762024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /inventory sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11772024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /network-bootstore-config sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11782024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /network-bootstore-config sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11792024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /omicron-physical-disks sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11802024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /omicron-physical-disks sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11812024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /omicron-zones sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11822024-12-20T21:46:22.528ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /sled-identifiers sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11832024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /sled-role sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11842024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /sleds sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11852024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /support/dladm-info sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11862024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /support/ipadm-info sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11872024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /support/zoneadm-info sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11882024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /support-bundles/{zpool_id}/{dataset_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11892024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = DELETE path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11902024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11912024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11922024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11932024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11942024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11952024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11962024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11972024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /switch-ports sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11982024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /timesync sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
11992024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /update sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12002024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = DELETE path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12012024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12022024-12-20T21:46:22.529ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /v2p sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12032024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = DELETE path = /vmms/{propolis_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12042024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /vmms/{propolis_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12052024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12062024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12072024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12082024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /vmms/{propolis_id}/poke sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12092024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12102024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12112024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /vmms/{propolis_id}/state sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12122024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /vmms/{propolis_id}/state sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12132024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12142024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /vpc-routes sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12152024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /vpc-routes sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12162024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /zones sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12172024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /zones/bundle-cleanup sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12182024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /zones/bundle-cleanup/context sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12192024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = PUT path = /zones/bundle-cleanup/context sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12202024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /zones/bundle-cleanup/utilization sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12212024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /zones/bundles sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12222024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /zones/bundles/{zone_name} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12232024-12-20T21:46:22.530ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = POST path = /zones/bundles/{zone_name} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12242024-12-20T21:46:22.531ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12252024-12-20T21:46:22.531ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12262024-12-20T21:46:22.531ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:34315 method = GET path = /zpools sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12272024-12-20T21:46:22.531ZINFOtest_mgs_metrics (dropshot): listening local_addr = [::1]:34315 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12282024-12-20T21:46:22.531ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:34315 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12292024-12-20T21:46:22.531ZDEBGtest_mgs_metrics (omicron_sled_agent::sim::Server): contacting server nexus sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12302024-12-20T21:46:22.531ZDEBGtest_mgs_metrics (NexusClient): client request body = Some(Body) method = POST sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = http://127.0.0.1:38423/sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
12312024-12-20T21:46:22.531ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12322024-12-20T21:46:22.531ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12332024-12-20T21:46:22.531ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12342024-12-20T21:46:22.594ZINFOtest_mgs_metrics (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
12352024-12-20T21:46:22.594ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:34047/vpc-routes
12362024-12-20T21:46:22.594ZWARNtest_mgs_metrics (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone
12372024-12-20T21:46:22.594ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 1.192278165s iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
12382024-12-20T21:46:22.595ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
12392024-12-20T21:46:22.595ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12402024-12-20T21:46:22.595ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12412024-12-20T21:46:22.596ZTRACtest_mgs_metrics (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = start saga failed total_found = 0
12422024-12-20T21:46:22.596ZWARNtest_mgs_metrics (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
12432024-12-20T21:46:22.596ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 1.181092816s iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
12442024-12-20T21:46:22.596ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12452024-12-20T21:46:22.600ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
12462024-12-20T21:46:22.600ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
12472024-12-20T21:46:22.600ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
12482024-12-20T21:46:22.601ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
12492024-12-20T21:46:22.601ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12502024-12-20T21:46:22.601ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12512024-12-20T21:46:22.602ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12522024-12-20T21:46:22.603ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54974
12532024-12-20T21:46:22.603ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34047 remote_addr = [::1]:41254 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12542024-12-20T21:46:22.604ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54974 req_id = 6f948f74-701a-493a-b75c-cc2ea33ac9c6 uri = /sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
12552024-12-20T21:46:22.604ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = GET remote_addr = [::1]:41254 req_id = c1c45f8f-0e0b-4832-a360-aece4b8f0c50 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
12562024-12-20T21:46:22.604ZINFOtest_mgs_metrics (nexus): registered sled agent sled_uuid = 039be560-54cc-49e3-88df-1a29dadbf913
12572024-12-20T21:46:22.605ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 773 local_addr = [::1]:34047 method = GET remote_addr = [::1]:41254 req_id = c1c45f8f-0e0b-4832-a360-aece4b8f0c50 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
12582024-12-20T21:46:22.605ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:34047/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1c45f8f-0e0b-4832-a360-aece4b8f0c50", "content-length": "2", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
12592024-12-20T21:46:22.605ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
12602024-12-20T21:46:22.606ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12612024-12-20T21:46:22.607ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12622024-12-20T21:46:22.614ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12632024-12-20T21:46:22.614ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12642024-12-20T21:46:22.615ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12652024-12-20T21:46:22.620ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
12662024-12-20T21:46:22.620ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
12672024-12-20T21:46:22.621ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d8 0f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53325 req_id = b811ce46-93eb-4f5c-a6d1-266517bf7ba1
12682024-12-20T21:46:22.621ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55311,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53325 req_id = b811ce46-93eb-4f5c-a6d1-266517bf7ba1
12692024-12-20T21:46:22.621ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55311, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53325 req_id = b811ce46-93eb-4f5c-a6d1-266517bf7ba1
12702024-12-20T21:46:22.622ZWARNtest_mgs_metrics (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
12712024-12-20T21:46:22.628ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 241.503172ms iteration = 2 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
12722024-12-20T21:46:22.628ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
12732024-12-20T21:46:22.628ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
12742024-12-20T21:46:22.629ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12752024-12-20T21:46:22.629ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12762024-12-20T21:46:22.629ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12772024-12-20T21:46:22.630ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Signaled
12782024-12-20T21:46:22.630ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
12792024-12-20T21:46:22.630ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12802024-12-20T21:46:22.630ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
12812024-12-20T21:46:22.630ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
12822024-12-20T21:46:22.631ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
12832024-12-20T21:46:22.631ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 27470 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54974 req_id = 6f948f74-701a-493a-b75c-cc2ea33ac9c6 response_code = 204 uri = /sled-agents/039be560-54cc-49e3-88df-1a29dadbf913
12842024-12-20T21:46:22.632ZDEBGtest_mgs_metrics (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:38423/sled-agents/039be560-54cc-49e3-88df-1a29dadbf913", status: 204, headers: {"x-request-id": "6f948f74-701a-493a-b75c-cc2ea33ac9c6", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} }) sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12852024-12-20T21:46:22.643ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12862024-12-20T21:46:22.644ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12872024-12-20T21:46:22.644ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12882024-12-20T21:46:22.645ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12892024-12-20T21:46:22.646ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12902024-12-20T21:46:22.646ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12912024-12-20T21:46:22.647ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12922024-12-20T21:46:22.647ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12932024-12-20T21:46:22.647ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12942024-12-20T21:46:22.647ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12952024-12-20T21:46:22.647ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12962024-12-20T21:46:22.647ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12972024-12-20T21:46:22.647ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45053 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
12982024-12-20T21:46:22.647ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45053 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
12992024-12-20T21:46:22.647ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45053 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13002024-12-20T21:46:22.647ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45053 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13012024-12-20T21:46:22.648ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13022024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13032024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13042024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13052024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13062024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13072024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13082024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13092024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36257 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13102024-12-20T21:46:22.649ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:36257 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13112024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:36257 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13122024-12-20T21:46:22.649ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:36257 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13132024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13142024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13152024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13162024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13172024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13182024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13192024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13202024-12-20T21:46:22.649ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13212024-12-20T21:46:22.650ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45607 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13222024-12-20T21:46:22.650ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45607 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13232024-12-20T21:46:22.650ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45607 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13242024-12-20T21:46:22.650ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45607 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13252024-12-20T21:46:22.655ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13262024-12-20T21:46:22.656ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13272024-12-20T21:46:22.657ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13282024-12-20T21:46:22.657ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13292024-12-20T21:46:22.658ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13302024-12-20T21:46:22.659ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13312024-12-20T21:46:22.661ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13322024-12-20T21:46:22.661ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13332024-12-20T21:46:22.661ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:45757 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13342024-12-20T21:46:22.661ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:45757 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13352024-12-20T21:46:22.662ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:45757 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13362024-12-20T21:46:22.662ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:45757 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13372024-12-20T21:46:22.662ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13382024-12-20T21:46:22.662ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13392024-12-20T21:46:22.662ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13402024-12-20T21:46:22.662ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13412024-12-20T21:46:22.662ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13422024-12-20T21:46:22.662ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13432024-12-20T21:46:22.663ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13442024-12-20T21:46:22.663ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13452024-12-20T21:46:22.663ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:33671 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13462024-12-20T21:46:22.663ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:33671 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13472024-12-20T21:46:22.663ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:33671 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13482024-12-20T21:46:22.663ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:33671 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13492024-12-20T21:46:22.665ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13502024-12-20T21:46:22.666ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13512024-12-20T21:46:22.666ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13522024-12-20T21:46:22.668ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13532024-12-20T21:46:22.668ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13542024-12-20T21:46:22.669ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13552024-12-20T21:46:22.669ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13562024-12-20T21:46:22.669ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13572024-12-20T21:46:22.670ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:44787 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13582024-12-20T21:46:22.670ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:44787 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13592024-12-20T21:46:22.670ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:44787 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13602024-12-20T21:46:22.670ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:44787 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13612024-12-20T21:46:22.672ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13622024-12-20T21:46:22.673ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13632024-12-20T21:46:22.673ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13642024-12-20T21:46:22.673ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13652024-12-20T21:46:22.675ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13662024-12-20T21:46:22.676ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13672024-12-20T21:46:22.676ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13682024-12-20T21:46:22.676ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13692024-12-20T21:46:22.676ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:38129 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13702024-12-20T21:46:22.676ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:38129 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13712024-12-20T21:46:22.676ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:38129 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13722024-12-20T21:46:22.676ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:38129 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13732024-12-20T21:46:22.676ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13742024-12-20T21:46:22.676ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13752024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13762024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13772024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13782024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13792024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13802024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13812024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34539 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13822024-12-20T21:46:22.677ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:34539 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13832024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:34539 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13842024-12-20T21:46:22.677ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:34539 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13852024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13862024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13872024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13882024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13892024-12-20T21:46:22.677ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13902024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13912024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13922024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13932024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:35881 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13942024-12-20T21:46:22.678ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:35881 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13952024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:35881 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13962024-12-20T21:46:22.678ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:35881 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
13972024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = GET path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13982024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = POST path = /crucible/0/regions server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
13992024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = DELETE path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
14002024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = GET path = /crucible/0/regions/{id} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
14012024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = GET path = /crucible/0/regions/{id}/snapshots server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
14022024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
14032024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
14042024-12-20T21:46:22.678ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
14052024-12-20T21:46:22.680ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:41781 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 versions = all
14062024-12-20T21:46:22.680ZINFOtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:41781 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
14072024-12-20T21:46:22.680ZDEBGtest_mgs_metrics (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:41781 server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
14082024-12-20T21:46:22.680ZINFOtest_mgs_metrics (SledAgent): Created Simulated Crucible Server address = [::1]:41781 kind = storage server = 039be560-54cc-49e3-88df-1a29dadbf913 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
14092024-12-20T21:46:22.680ZDEBGtest_mgs_metrics (init_with_steps): Running step start_crucible_pantry
14102024-12-20T21:46:22.687ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = GET path = /crucible/pantry/0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14112024-12-20T21:46:22.690ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = GET path = /crucible/pantry/0/job/{id}/is-finished sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14122024-12-20T21:46:22.691ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = GET path = /crucible/pantry/0/job/{id}/ok sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14132024-12-20T21:46:22.691ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = DELETE path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14142024-12-20T21:46:22.692ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = GET path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14152024-12-20T21:46:22.692ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = POST path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14162024-12-20T21:46:22.692ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = POST path = /crucible/pantry/0/volume/{id}/background sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14172024-12-20T21:46:22.692ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = POST path = /crucible/pantry/0/volume/{id}/bulk-write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14182024-12-20T21:46:22.692ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = POST path = /crucible/pantry/0/volume/{id}/import-from-url sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14192024-12-20T21:46:22.693ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = POST path = /crucible/pantry/0/volume/{id}/scrub sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14202024-12-20T21:46:22.693ZDEBGtest_mgs_metrics (dropshot): registered endpoint kind = pantry local_addr = [::1]:39291 method = POST path = /crucible/pantry/0/volume/{id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
14212024-12-20T21:46:22.693ZINFOtest_mgs_metrics (dropshot): listening kind = pantry local_addr = [::1]:39291 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14222024-12-20T21:46:22.693ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes kind = pantry local_addr = [::1]:39291 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14232024-12-20T21:46:22.693ZINFOtest_mgs_metrics (omicron_sled_agent::sim::Server): Started Simulated Crucible Pantry address = [::1]:39291 kind = pantry sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14242024-12-20T21:46:22.694ZDEBGtest_mgs_metrics (init_with_steps): Running step populate_internal_dns
14252024-12-20T21:46:22.694ZDEBGtest_mgs_metrics: Populating Internal DNS
14262024-12-20T21:46:22.746ZINFOtest_mgs_metrics: DNS population: DnsConfigParams { generation: Generation( 1, ), time_created: 2024-12-20T21:46:22.746507024Z, zones: [ DnsConfigZone { zone_name: "control-plane.oxide.internal", records: { "_nameservice._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 34419, target: "b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal", }, ), ], "_dendrite._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 36515, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 35701, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_mgd._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 37229, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 35923, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176.host": [ Aaaa( ::ffff:127.0.0.1, ), ], "812f8fd8-a5c9-4745-bb3e-90192659f489.host": [ Aaaa( ::1, ), ], "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host": [ Aaaa( ::1, ), ], "_nexus._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 38423, target: "cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176.host.control-plane.oxide.internal", }, ), ], "_clickhouse-native._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "812f8fd8-a5c9-4745-bb3e-90192659f489.host.control-plane.oxide.internal", }, ), ], "065de285-8587-4a14-900b-e3925b69df4a.host": [ Aaaa( ::1, ), ], "_clickhouse._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 36651, target: "812f8fd8-a5c9-4745-bb3e-90192659f489.host.control-plane.oxide.internal", }, ), ], "b83e9e5e-6a54-4940-bca6-01e138d611f0.host": [ Aaaa( ::1, ), ], "_external-dns._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 40731, target: "67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal", }, ), ], "67c5f200-c220-4da8-95cb-9522611a1492.host": [ Aaaa( ::1, ), ], "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host": [ Aaaa( ::1, ), ], "_crucible-pantry._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 39291, target: "bca45c56-47e5-420e-82e5-fce0647d0ba3.host.control-plane.oxide.internal", }, ), ], "_cockroach._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 43309, target: "065de285-8587-4a14-900b-e3925b69df4a.host.control-plane.oxide.internal", }, ), ], "bca45c56-47e5-420e-82e5-fce0647d0ba3.host": [ Aaaa( ::1, ), ], "_mgs._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 36481, target: "dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal", }, ), Srv( Srv { prio: 0, weight: 0, port: 45883, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], }, }, ], }
14272024-12-20T21:46:22.754ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:34419/config
14282024-12-20T21:46:22.755ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14292024-12-20T21:46:22.759ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
14302024-12-20T21:46:22.759ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
14312024-12-20T21:46:22.759ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14322024-12-20T21:46:22.774ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14332024-12-20T21:46:22.774ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14342024-12-20T21:46:22.774ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14352024-12-20T21:46:22.774ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14362024-12-20T21:46:22.774ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14372024-12-20T21:46:22.793ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:34419 remote_addr = [::1]:58408
14382024-12-20T21:46:22.794ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:34419 method = PUT remote_addr = [::1]:58408 req_id = 3bcfa485-8f7d-4d77-b9c0-dcd5eea96d2c uri = /config
14392024-12-20T21:46:22.795ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 1 req_id = 3bcfa485-8f7d-4d77-b9c0-dcd5eea96d2c
14402024-12-20T21:46:22.795ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
14412024-12-20T21:46:22.795ZDEBGtest_mgs_metrics (store): creating tree tree_name = generation_1_zone_control-plane.oxide.internal
14422024-12-20T21:46:22.797ZDEBGtest_mgs_metrics (store): updating current config
14432024-12-20T21:46:22.806ZDEBGtest_mgs_metrics (store): flushing default tree
14442024-12-20T21:46:22.808ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
14452024-12-20T21:46:22.809ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
14462024-12-20T21:46:22.810ZINFOtest_mgs_metrics (store): pruning trees for generations older than 1 keep = 3
14472024-12-20T21:46:22.810ZINFOtest_mgs_metrics (store): updated generation new_generation = 1 req_id = 3bcfa485-8f7d-4d77-b9c0-dcd5eea96d2c
14482024-12-20T21:46:22.810ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 15838 local_addr = [::1]:34419 method = PUT remote_addr = [::1]:58408 req_id = 3bcfa485-8f7d-4d77-b9c0-dcd5eea96d2c response_code = 204 uri = /config
14492024-12-20T21:46:22.810ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:34419/config", status: 204, headers: {"x-request-id": "3bcfa485-8f7d-4d77-b9c0-dcd5eea96d2c", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
14502024-12-20T21:46:22.811ZDEBGtest_mgs_metrics (init_with_steps): Running step configure_sled_agent1
14512024-12-20T21:46:22.854ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:34047/omicron-zones
14522024-12-20T21:46:22.865ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b5 04 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53850 req_id = 3088b638-8e2d-4132-9cec-973c5ff5eed4
14532024-12-20T21:46:22.865ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46340,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53850 req_id = 3088b638-8e2d-4132-9cec-973c5ff5eed4
14542024-12-20T21:46:22.868ZDEBGtest_mgs_metrics (store): query key key = _mgs._tcp
14552024-12-20T21:46:22.874ZDEBGtest_mgs_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
14562024-12-20T21:46:22.874ZDEBGtest_mgs_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
14572024-12-20T21:46:22.874ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53850 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36481, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 45883, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3088b638-8e2d-4132-9cec-973c5ff5eed4
14582024-12-20T21:46:22.874ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14592024-12-20T21:46:22.927ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
14602024-12-20T21:46:22.927ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
14612024-12-20T21:46:22.927ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
14622024-12-20T21:46:22.927ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14632024-12-20T21:46:22.928ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14642024-12-20T21:46:22.928ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14652024-12-20T21:46:22.930ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14662024-12-20T21:46:22.934ZINFOtest_mgs_metrics (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
14672024-12-20T21:46:22.941ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:34047/eip-gateways
14682024-12-20T21:46:22.941ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
14692024-12-20T21:46:22.942ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14702024-12-20T21:46:22.942ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
14712024-12-20T21:46:22.942ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14722024-12-20T21:46:22.943ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14732024-12-20T21:46:22.944ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34047 remote_addr = [::1]:41256 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
14742024-12-20T21:46:22.945ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36481, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 45883, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2106, tv_nsec: 728734783 } })
14752024-12-20T21:46:22.948ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
14762024-12-20T21:46:22.948ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
14772024-12-20T21:46:22.948ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41254 req_id = c8e594d4-7c37-4c5f-895c-1735f43669ae sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
14782024-12-20T21:46:22.948ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14792024-12-20T21:46:22.948ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14802024-12-20T21:46:22.948ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14812024-12-20T21:46:22.952ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14822024-12-20T21:46:22.953ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 6505 local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41254 req_id = c8e594d4-7c37-4c5f-895c-1735f43669ae response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
14832024-12-20T21:46:22.956ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14842024-12-20T21:46:22.961ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14852024-12-20T21:46:22.962ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
14862024-12-20T21:46:22.962ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14872024-12-20T21:46:22.962ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
14882024-12-20T21:46:22.962ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14892024-12-20T21:46:22.962ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14902024-12-20T21:46:22.962ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14912024-12-20T21:46:22.962ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14922024-12-20T21:46:22.962ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14932024-12-20T21:46:22.963ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14942024-12-20T21:46:22.966ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14952024-12-20T21:46:22.972ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14962024-12-20T21:46:22.972ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
14972024-12-20T21:46:22.973ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14982024-12-20T21:46:22.973ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14992024-12-20T21:46:22.974ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15002024-12-20T21:46:22.984ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15012024-12-20T21:46:22.984ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15022024-12-20T21:46:22.984ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15032024-12-20T21:46:22.985ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15042024-12-20T21:46:22.987ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:34047/eip-gateways", status: 204, headers: {"x-request-id": "c8e594d4-7c37-4c5f-895c-1735f43669ae", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
15052024-12-20T21:46:22.988ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 1.571873601s iteration = 1 status = {}
15062024-12-20T21:46:22.990ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41256 req_id = c1165707-727b-44d0-90c1-4740ac0ec8e0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-zones
15072024-12-20T21:46:22.991ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 92 71 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .q..........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54051 req_id = 6e81f5ce-ad9f-471b-b2fe-b5ef36880b87
15082024-12-20T21:46:22.991ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37489,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54051 req_id = 6e81f5ce-ad9f-471b-b2fe-b5ef36880b87
15092024-12-20T21:46:22.991ZDEBGtest_mgs_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
15102024-12-20T21:46:22.991ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54051 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6e81f5ce-ad9f-471b-b2fe-b5ef36880b87
15112024-12-20T21:46:22.992ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d2 c2 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52309 req_id = c11e2e9e-e694-4fa7-8f60-7169e48d3866
15122024-12-20T21:46:22.992ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53954,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52309 req_id = c11e2e9e-e694-4fa7-8f60-7169e48d3866
15132024-12-20T21:46:22.992ZDEBGtest_mgs_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
15142024-12-20T21:46:22.992ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52309 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c11e2e9e-e694-4fa7-8f60-7169e48d3866
15152024-12-20T21:46:23.065ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager body = None method = GET uri = http://[::1]:34315/v2p
15162024-12-20T21:46:23.067ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 77007 local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41256 req_id = c1165707-727b-44d0-90c1-4740ac0ec8e0 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-zones
15172024-12-20T21:46:23.068ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15182024-12-20T21:46:23.068ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15192024-12-20T21:46:23.068ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15202024-12-20T21:46:23.148ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
15212024-12-20T21:46:23.149ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34315 remote_addr = [::1]:57964 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
15222024-12-20T21:46:23.151ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:34047/omicron-zones", status: 204, headers: {"x-request-id": "c1165707-727b-44d0-90c1-4740ac0ec8e0", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
15232024-12-20T21:46:23.152ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:34047/network-bootstore-config
15242024-12-20T21:46:23.152ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34315 method = GET remote_addr = [::1]:57964 req_id = 4d211407-7be7-4c66-8e70-421a55198da1 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
15252024-12-20T21:46:23.156ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 4187 local_addr = [::1]:34315 method = GET remote_addr = [::1]:57964 req_id = 4d211407-7be7-4c66-8e70-421a55198da1 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /v2p
15262024-12-20T21:46:23.156ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15272024-12-20T21:46:23.156ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15282024-12-20T21:46:23.157ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15292024-12-20T21:46:23.157ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41256 req_id = f8455203-b11e-4953-bda4-cbf721259697 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
15302024-12-20T21:46:23.158ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 15 db 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61392 req_id = 3ca21695-040f-463a-8f90-a1f1565e5d80
15312024-12-20T21:46:23.158ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5595,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61392 req_id = 3ca21695-040f-463a-8f90-a1f1565e5d80
15322024-12-20T21:46:23.158ZDEBGtest_mgs_metrics (store): query key key = _clickhouse-admin-keeper._tcp
15332024-12-20T21:46:23.158ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5595, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61392 req_id = 3ca21695-040f-463a-8f90-a1f1565e5d80
15342024-12-20T21:46:23.163ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = 039be560-54cc-49e3-88df-1a29dadbf913 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:34315/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d211407-7be7-4c66-8e70-421a55198da1", "content-length": "2", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
15352024-12-20T21:46:23.164ZINFOtest_mgs_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-039be560
15362024-12-20T21:46:23.164ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-039be560
15372024-12-20T21:46:23.164ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-039be560
15382024-12-20T21:46:23.165ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:34047/v2p
15392024-12-20T21:46:23.166ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 8703 local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41256 req_id = f8455203-b11e-4953-bda4-cbf721259697 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
15402024-12-20T21:46:23.167ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection background_task = inventory_collection
15412024-12-20T21:46:23.167ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:36481
15422024-12-20T21:46:23.167ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/ignition
15432024-12-20T21:46:23.167ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34047 remote_addr = [::1]:41258 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
15442024-12-20T21:46:23.172ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15452024-12-20T21:46:23.172ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15462024-12-20T21:46:23.173ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15472024-12-20T21:46:23.173ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:34047/network-bootstore-config", status: 204, headers: {"x-request-id": "f8455203-b11e-4953-bda4-cbf721259697", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
15482024-12-20T21:46:23.199ZDEBGtest_mgs_metrics (init_with_steps): Running step configure_sled_agent2
15492024-12-20T21:46:23.284ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:34315/omicron-zones
15502024-12-20T21:46:23.285ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = GET remote_addr = [::1]:41258 req_id = 63476ebe-4fbb-4e77-bf3a-d8fcba741ade sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
15512024-12-20T21:46:23.293ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 8657 local_addr = [::1]:34047 method = GET remote_addr = [::1]:41258 req_id = 63476ebe-4fbb-4e77-bf3a-d8fcba741ade response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
15522024-12-20T21:46:23.295ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34315 remote_addr = [::1]:57966 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
15532024-12-20T21:46:23.298ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:34047/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "63476ebe-4fbb-4e77-bf3a-d8fcba741ade", "content-length": "2", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
15542024-12-20T21:46:23.298ZINFOtest_mgs_metrics (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
15552024-12-20T21:46:23.298ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
15562024-12-20T21:46:23.298ZINFOtest_mgs_metrics (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
15572024-12-20T21:46:23.299ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 1.89116258s iteration = 1 status = {}
15582024-12-20T21:46:23.300ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15592024-12-20T21:46:23.300ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15602024-12-20T21:46:23.300ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15612024-12-20T21:46:23.307ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34315 method = PUT remote_addr = [::1]:57966 req_id = 7ae4d0b0-6eef-4001-a609-450599a19f2d sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /omicron-zones
15622024-12-20T21:46:23.312ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 5205 local_addr = [::1]:34315 method = PUT remote_addr = [::1]:57966 req_id = 7ae4d0b0-6eef-4001-a609-450599a19f2d response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /omicron-zones
15632024-12-20T21:46:23.312ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
15642024-12-20T21:46:23.312ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15652024-12-20T21:46:23.312ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15662024-12-20T21:46:23.313ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15672024-12-20T21:46:23.313ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15682024-12-20T21:46:23.316ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:34315/omicron-zones", status: 204, headers: {"x-request-id": "7ae4d0b0-6eef-4001-a609-450599a19f2d", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
15692024-12-20T21:46:23.316ZDEBGtest_mgs_metrics: client request body = Some(Body) method = PUT uri = http://[::1]:34315/network-bootstore-config
15702024-12-20T21:46:23.318ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5313db8c-9a4c-4cff-b0e0-d4f16d6ded1b", "content-length": "835", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
15712024-12-20T21:46:23.319ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0
15722024-12-20T21:46:23.319ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15732024-12-20T21:46:23.319ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15742024-12-20T21:46:23.319ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15752024-12-20T21:46:23.319ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15762024-12-20T21:46:23.322ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34315 method = PUT remote_addr = [::1]:57966 req_id = f09aa151-9472-4a36-b48b-fa6003007621 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /network-bootstore-config
15772024-12-20T21:46:23.322ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 2066 local_addr = [::1]:34315 method = PUT remote_addr = [::1]:57966 req_id = f09aa151-9472-4a36-b48b-fa6003007621 response_code = 204 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /network-bootstore-config
15782024-12-20T21:46:23.326ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15792024-12-20T21:46:23.326ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15802024-12-20T21:46:23.326ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15812024-12-20T21:46:23.327ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15822024-12-20T21:46:23.327ZDEBGtest_mgs_metrics: client response result = Ok(Response { url: "http://[::1]:34315/network-bootstore-config", status: 204, headers: {"x-request-id": "f09aa151-9472-4a36-b48b-fa6003007621", "date": "Fri, 20 Dec 2024 21:46:22 GMT"} })
15832024-12-20T21:46:23.329ZDEBGtest_mgs_metrics (init_with_steps): Running step start_nexus_external
15842024-12-20T21:46:23.329ZDEBGtest_mgs_metrics: Starting Nexus (external API)
15852024-12-20T21:46:24.157ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15862024-12-20T21:46:24.157ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15872024-12-20T21:46:24.157ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15882024-12-20T21:46:24.157ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
15892024-12-20T21:46:24.158ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15902024-12-20T21:46:24.170ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15912024-12-20T21:46:24.178ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
15922024-12-20T21:46:24.182ZTRACtest_mgs_metrics (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
15932024-12-20T21:46:24.182ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
15942024-12-20T21:46:24.182ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15952024-12-20T21:46:24.182ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
15962024-12-20T21:46:24.182ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15972024-12-20T21:46:24.182ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15982024-12-20T21:46:24.184ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15992024-12-20T21:46:24.187ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16002024-12-20T21:46:24.187ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16012024-12-20T21:46:24.188ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16022024-12-20T21:46:24.188ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16032024-12-20T21:46:24.196ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "71013c98-699e-469a-9a36-4efd657ff7a1", "content-length": "734", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16042024-12-20T21:46:24.197ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/sp/caboose?firmware_slot=0
16052024-12-20T21:46:24.197ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16062024-12-20T21:46:24.197ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16072024-12-20T21:46:24.197ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16082024-12-20T21:46:24.198ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16092024-12-20T21:46:24.206ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16102024-12-20T21:46:24.206ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16112024-12-20T21:46:24.207ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16122024-12-20T21:46:24.212ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16132024-12-20T21:46:24.212ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16142024-12-20T21:46:24.212ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16152024-12-20T21:46:24.212ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16162024-12-20T21:46:24.217ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16172024-12-20T21:46:24.226ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16182024-12-20T21:46:24.227ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16192024-12-20T21:46:24.227ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16202024-12-20T21:46:24.227ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16212024-12-20T21:46:24.248ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
16222024-12-20T21:46:24.249ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet
16232024-12-20T21:46:24.249ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16242024-12-20T21:46:24.249ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16252024-12-20T21:46:24.249ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16262024-12-20T21:46:24.249ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16272024-12-20T21:46:24.254ZINFOtest_mgs_metrics (InternalApi): set cockroachdb setting setting = cluster.preserve_downgrade_option value = 22.1
16282024-12-20T21:46:24.254ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16292024-12-20T21:46:24.254ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16302024-12-20T21:46:24.255ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16312024-12-20T21:46:24.257ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16322024-12-20T21:46:24.261ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
16332024-12-20T21:46:24.261ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
16342024-12-20T21:46:24.261ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16352024-12-20T21:46:24.261ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16362024-12-20T21:46:24.264ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
16372024-12-20T21:46:24.265ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16382024-12-20T21:46:24.265ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16392024-12-20T21:46:24.265ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16402024-12-20T21:46:24.265ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16412024-12-20T21:46:24.275ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16422024-12-20T21:46:24.281ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
16432024-12-20T21:46:24.281ZDEBGtest_mgs_metrics (InternalApi): updating rack subnet for rack c19a698f-c6f9-4a17-ae30-20d711b8f7dc to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), )
16442024-12-20T21:46:24.281ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16452024-12-20T21:46:24.281ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16462024-12-20T21:46:24.282ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16472024-12-20T21:46:24.292ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f34b306a-fb42-4ada-8143-984938046b10", "content-length": "111", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16482024-12-20T21:46:24.293ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/sp/caboose?firmware_slot=1
16492024-12-20T21:46:24.296ZINFOtest_mgs_metrics (InternalApi): Using static configuration for external switchports
16502024-12-20T21:46:24.296ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16512024-12-20T21:46:24.296ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16522024-12-20T21:46:24.298ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16532024-12-20T21:46:24.300ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bc9b8d6-01df-473f-b14d-3df99f700b4e", "content-length": "111", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16542024-12-20T21:46:24.303ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/caboose?firmware_slot=0
16552024-12-20T21:46:24.303ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "69652b9a-b06b-49c5-947e-4f3fd8fb4075", "content-length": "167", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16562024-12-20T21:46:24.303ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/caboose?firmware_slot=1
16572024-12-20T21:46:24.304ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dd3068d-8f0f-48ad-b8fa-5f5dcf697588", "content-length": "167", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16582024-12-20T21:46:24.305ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/stage0/caboose?firmware_slot=0
16592024-12-20T21:46:24.313ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fa4bfdf-5069-4e1c-95c6-cb5339df46cc", "content-length": "175", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16602024-12-20T21:46:24.314ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/stage0/caboose?firmware_slot=1
16612024-12-20T21:46:24.317ZTRACtest_mgs_metrics (service-firewall-rule-progator): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-12-20T21:23:45.899574Z, time_modified: 2024-12-20T21:23:45.899574Z, allowed_ips: None } background_task = service_firewall_rule_propagation
16622024-12-20T21:46:24.317ZDEBGtest_mgs_metrics (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
16632024-12-20T21:46:24.317ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
16642024-12-20T21:46:24.317ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
16652024-12-20T21:46:24.317ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71d1df94-f486-43cf-8c6d-d70eebb40208", "content-length": "175", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16662024-12-20T21:46:24.320ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/cmpa
16672024-12-20T21:46:24.323ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6151d9eb-b0f1-4448-bc4a-60fdd53ab6d3", "content-length": "702", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16682024-12-20T21:46:24.323ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/cfpa
16692024-12-20T21:46:24.324ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a12a266-3ee5-45ed-ac32-f7b8ec93b371", "content-length": "727", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16702024-12-20T21:46:24.324ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/cfpa
16712024-12-20T21:46:24.325ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "988b2fbf-68b2-43a3-a0f5-861a4afc1ca2", "content-length": "729", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16722024-12-20T21:46:24.326ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/cfpa
16732024-12-20T21:46:24.327ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "245bf198-434b-4e96-8998-f1a68e391635", "content-length": "728", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16742024-12-20T21:46:24.327ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1
16752024-12-20T21:46:24.367ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "edb008c3-4747-4a9b-99b1-6dd3e2a69ef6", "content-length": "734", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16762024-12-20T21:46:24.368ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/sp/caboose?firmware_slot=0
16772024-12-20T21:46:24.373ZINFOtest_mgs_metrics (InternalApi): Recording Rack Network Configuration
16782024-12-20T21:46:24.374ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16792024-12-20T21:46:24.374ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
16802024-12-20T21:46:24.374ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16812024-12-20T21:46:24.376ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8209eb17-202c-46b6-8746-d9ef86820cb6", "content-length": "111", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16822024-12-20T21:46:24.379ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/sp/caboose?firmware_slot=1
16832024-12-20T21:46:24.384ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "75b1e36b-c5c7-4e72-80aa-346e56f5e6b5", "content-length": "111", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16842024-12-20T21:46:24.384ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/caboose?firmware_slot=0
16852024-12-20T21:46:24.399ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "636066a8-d66a-4cfe-820d-4116ccf49f40", "content-length": "167", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16862024-12-20T21:46:24.399ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/caboose?firmware_slot=1
16872024-12-20T21:46:24.399ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "65ef3059-7f89-4d31-8ebc-a1cd50eaa228", "content-length": "167", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16882024-12-20T21:46:24.399ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/stage0/caboose?firmware_slot=0
16892024-12-20T21:46:24.399ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fba4a3dd-3395-49e7-9956-c7b5f02d15ac", "content-length": "175", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16902024-12-20T21:46:24.399ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/stage0/caboose?firmware_slot=1
16912024-12-20T21:46:24.409ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "068b4587-5b9f-4b30-be21-8d496dad3e53", "content-length": "175", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16922024-12-20T21:46:24.409ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/cmpa
16932024-12-20T21:46:24.412ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e636f2a1-9889-41bc-a8e7-b079869bcc01", "content-length": "702", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16942024-12-20T21:46:24.413ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/cfpa
16952024-12-20T21:46:24.413ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44f6fc69-fc1b-4b1c-aef9-032cf978cda7", "content-length": "727", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16962024-12-20T21:46:24.413ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/cfpa
16972024-12-20T21:46:24.413ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "90b312b9-58fb-4488-993b-4e7eebb746af", "content-length": "729", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
16982024-12-20T21:46:24.414ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/cfpa
16992024-12-20T21:46:24.426ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb060258-88d5-4832-a566-c17679eb2e9d", "content-length": "728", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17002024-12-20T21:46:24.427ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0
17012024-12-20T21:46:24.429ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd92b3f9-ec53-4320-877c-6c1d1d0a03a0", "content-length": "723", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17022024-12-20T21:46:24.431ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/sp/caboose?firmware_slot=0
17032024-12-20T21:46:24.431ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1042c3d6-3150-4d8a-bcff-4577f12a2513", "content-length": "109", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17042024-12-20T21:46:24.431ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/sp/caboose?firmware_slot=1
17052024-12-20T21:46:24.433ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6c44536-4d67-422a-be39-91dd9a4f1acd", "content-length": "109", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17062024-12-20T21:46:24.433ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/caboose?firmware_slot=0
17072024-12-20T21:46:24.466ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f119e799-ecc4-48f2-82c4-fe93c3f2b6ca", "content-length": "169", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17082024-12-20T21:46:24.466ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/caboose?firmware_slot=1
17092024-12-20T21:46:24.475ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7e7c505-da4e-4225-b99e-3502df6dd4ec", "content-length": "169", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17102024-12-20T21:46:24.475ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/stage0/caboose?firmware_slot=0
17112024-12-20T21:46:24.477ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "180b4a97-4610-4f58-9941-9eb5fbd6e79d", "content-length": "178", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17122024-12-20T21:46:24.477ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/stage0/caboose?firmware_slot=1
17132024-12-20T21:46:24.480ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8af5bc8-0dcc-4e9a-b7fd-a272c51654d3", "content-length": "178", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17142024-12-20T21:46:24.482ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/cmpa
17152024-12-20T21:46:24.482ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77484c8f-b0dc-4577-8469-98f4535b34ca", "content-length": "702", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17162024-12-20T21:46:24.482ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/cfpa
17172024-12-20T21:46:24.483ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f37fe86-c857-4185-bc1a-a947c82f6bdb", "content-length": "727", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17182024-12-20T21:46:24.483ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/cfpa
17192024-12-20T21:46:24.484ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7da0f186-ce8b-4ce1-b8a6-6217a23bfc66", "content-length": "729", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17202024-12-20T21:46:24.486ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/cfpa
17212024-12-20T21:46:24.487ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4528c46-a0fd-4742-810a-981c478f3eff", "content-length": "728", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17222024-12-20T21:46:24.487ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1
17232024-12-20T21:46:24.487ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "35cfa267-3df9-4cab-bb52-17b9baedf5c0", "content-length": "723", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17242024-12-20T21:46:24.487ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/sp/caboose?firmware_slot=0
17252024-12-20T21:46:24.490ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7c6fcc2-8889-4322-9b54-a5c3a8bffdec", "content-length": "109", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17262024-12-20T21:46:24.491ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/sp/caboose?firmware_slot=1
17272024-12-20T21:46:24.493ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a85585d6-da35-4582-b415-e6650dddc59c", "content-length": "109", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17282024-12-20T21:46:24.494ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/caboose?firmware_slot=0
17292024-12-20T21:46:24.508ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93529865-b80d-4dec-ad49-f3c212e21d47", "content-length": "169", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17302024-12-20T21:46:24.508ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/caboose?firmware_slot=1
17312024-12-20T21:46:24.510ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62bef85a-a975-4b5c-b591-1e1f4fd1d1de", "content-length": "169", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17322024-12-20T21:46:24.511ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/stage0/caboose?firmware_slot=0
17332024-12-20T21:46:24.513ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2616124-a7fe-4256-ad40-3bfbf6936caa", "content-length": "178", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17342024-12-20T21:46:24.513ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/stage0/caboose?firmware_slot=1
17352024-12-20T21:46:24.515ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3967d91b-4256-44b2-b96d-b8e54519d439", "content-length": "178", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17362024-12-20T21:46:24.515ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/cmpa
17372024-12-20T21:46:24.516ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a6d9ee0-20c7-45f2-a90a-314031bff8a6", "content-length": "702", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17382024-12-20T21:46:24.516ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/cfpa
17392024-12-20T21:46:24.519ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3503c2b-f566-48e9-974a-5f10c10d9db7", "content-length": "727", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17402024-12-20T21:46:24.519ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/cfpa
17412024-12-20T21:46:24.520ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06621b4a-5311-4f61-8889-78d11bfa9a61", "content-length": "729", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17422024-12-20T21:46:24.520ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/cfpa
17432024-12-20T21:46:24.521ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4147de05-2090-42ea-abe4-cfa463215686", "content-length": "728", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17442024-12-20T21:46:24.522ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:45883
17452024-12-20T21:46:24.522ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/ignition
17462024-12-20T21:46:24.523ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6f267bb-b58d-4e19-858a-98668ce9558a", "content-length": "835", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17472024-12-20T21:46:24.525ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/sp/switch/0
17482024-12-20T21:46:24.526ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ee5a7db-0b20-40cd-9778-db30c02a1d24", "content-length": "734", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17492024-12-20T21:46:24.526ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/sp/switch/1
17502024-12-20T21:46:24.526ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c930dcc3-d7d3-443e-9354-320ef8f2dc76", "content-length": "734", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17512024-12-20T21:46:24.527ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/sp/sled/0
17522024-12-20T21:46:24.528ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd2878b8-42cf-4b96-b1b6-55648ae506a9", "content-length": "723", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17532024-12-20T21:46:24.528ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/sp/sled/1
17542024-12-20T21:46:24.529ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90f0dbd0-3ebb-4b9d-89bd-494e07fdce7a", "content-length": "723", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
17552024-12-20T21:46:24.529ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
17562024-12-20T21:46:24.529ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17572024-12-20T21:46:24.532ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
17582024-12-20T21:46:24.532ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
17592024-12-20T21:46:24.532ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
17602024-12-20T21:46:24.533ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17612024-12-20T21:46:24.537ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
17622024-12-20T21:46:24.537ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
17632024-12-20T21:46:24.537ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17642024-12-20T21:46:24.537ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
17652024-12-20T21:46:24.537ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
17662024-12-20T21:46:24.538ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
17672024-12-20T21:46:24.571ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
17682024-12-20T21:46:24.573ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17692024-12-20T21:46:24.574ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17702024-12-20T21:46:24.574ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17712024-12-20T21:46:24.574ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17722024-12-20T21:46:24.574ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17732024-12-20T21:46:24.576ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
17742024-12-20T21:46:24.576ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
17752024-12-20T21:46:24.576ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
17762024-12-20T21:46:24.576ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
17772024-12-20T21:46:24.579ZDEBGtest_mgs_metrics (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
17782024-12-20T21:46:24.579ZDEBGtest_mgs_metrics (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
17792024-12-20T21:46:24.579ZINFOtest_mgs_metrics (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation
17802024-12-20T21:46:24.579ZDEBGtest_mgs_metrics (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 3.162942689s
17812024-12-20T21:46:24.579ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 3.1629828s iteration = 1 status = {}
17822024-12-20T21:46:24.583ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17832024-12-20T21:46:24.587ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17842024-12-20T21:46:24.592ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17852024-12-20T21:46:24.592ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17862024-12-20T21:46:24.592ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17872024-12-20T21:46:24.592ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 30952241-9bd9-4f3e-858e-dff5aebe8761, lookup_type: ByName("oxide-service-pool") }
17882024-12-20T21:46:24.592ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17892024-12-20T21:46:24.592ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17902024-12-20T21:46:24.592ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17912024-12-20T21:46:24.593ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17922024-12-20T21:46:24.614ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17932024-12-20T21:46:24.617ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 30952241-9bd9-4f3e-858e-dff5aebe8761, lookup_type: ByName("oxide-service-pool") } result = Ok(())
17942024-12-20T21:46:24.617ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17952024-12-20T21:46:24.617ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
17962024-12-20T21:46:24.618ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17972024-12-20T21:46:24.621ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 30952241-9bd9-4f3e-858e-dff5aebe8761, lookup_type: ByName("oxide-service-pool") }
17982024-12-20T21:46:24.621ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17992024-12-20T21:46:24.621ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18002024-12-20T21:46:24.621ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18012024-12-20T21:46:24.621ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18022024-12-20T21:46:24.624ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18032024-12-20T21:46:24.632ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 30952241-9bd9-4f3e-858e-dff5aebe8761, lookup_type: ByName("oxide-service-pool") } result = Ok(())
18042024-12-20T21:46:24.676ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 30952241-9bd9-4f3e-858e-dff5aebe8761, lookup_type: ByName("oxide-service-pool") }
18052024-12-20T21:46:24.676ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18062024-12-20T21:46:24.676ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18072024-12-20T21:46:24.677ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18082024-12-20T21:46:24.677ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18092024-12-20T21:46:24.680ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18102024-12-20T21:46:24.686ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 30952241-9bd9-4f3e-858e-dff5aebe8761, lookup_type: ByName("oxide-service-pool") } result = Ok(())
18112024-12-20T21:46:24.728ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:34315
18122024-12-20T21:46:24.728ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = http://[::1]:34315 background_task = inventory_collection body = None method = GET uri = http://[::1]:34315/inventory
18132024-12-20T21:46:24.729ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
18142024-12-20T21:46:24.730ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18152024-12-20T21:46:24.730ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18162024-12-20T21:46:24.731ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18172024-12-20T21:46:24.731ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18182024-12-20T21:46:24.731ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34315 remote_addr = [::1]:57968 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
18192024-12-20T21:46:24.731ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34315 method = GET remote_addr = [::1]:57968 req_id = 5031dc36-666f-4cb5-a9a8-1c59c4825a90 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
18202024-12-20T21:46:24.732ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 533 local_addr = [::1]:34315 method = GET remote_addr = [::1]:57968 req_id = 5031dc36-666f-4cb5-a9a8-1c59c4825a90 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
18212024-12-20T21:46:24.732ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = http://[::1]:34315 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:34315/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5031dc36-666f-4cb5-a9a8-1c59c4825a90", "content-length": "4237", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
18222024-12-20T21:46:24.782ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:34047
18232024-12-20T21:46:24.786ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = http://[::1]:34047 background_task = inventory_collection body = None method = GET uri = http://[::1]:34047/inventory
18242024-12-20T21:46:24.787ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18252024-12-20T21:46:24.795ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
18262024-12-20T21:46:24.795ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34047 remote_addr = [::1]:41260 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
18272024-12-20T21:46:24.796ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = GET remote_addr = [::1]:41260 req_id = eb81d57f-34f2-4b92-9452-c3b6e317c031 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
18282024-12-20T21:46:24.798ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 1054 local_addr = [::1]:34047 method = GET remote_addr = [::1]:41260 req_id = eb81d57f-34f2-4b92-9452-c3b6e317c031 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
18292024-12-20T21:46:24.799ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = http://[::1]:34047 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:34047/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb81d57f-34f2-4b92-9452-c3b6e317c031", "content-length": "6315", "date": "Fri, 20 Dec 2024 21:46:24 GMT"} })
18302024-12-20T21:46:24.799ZDEBGtest_mgs_metrics (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
18312024-12-20T21:46:24.799ZDEBGtest_mgs_metrics (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
18322024-12-20T21:46:24.799ZDEBGtest_mgs_metrics (BackgroundTasks): finished collection background_task = inventory_collection
18332024-12-20T21:46:24.804ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
18342024-12-20T21:46:24.804ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18352024-12-20T21:46:24.805ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18362024-12-20T21:46:24.805ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18372024-12-20T21:46:24.805ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18382024-12-20T21:46:24.805ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18392024-12-20T21:46:24.809ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
18402024-12-20T21:46:24.809ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18412024-12-20T21:46:24.809ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18422024-12-20T21:46:24.810ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18432024-12-20T21:46:25.289ZINFOtest_mgs_metrics (InternalApi): inserted blueprint blueprint_id = fe9cda94-6af2-4b74-82e6-5d0f7bdb4cc9
18442024-12-20T21:46:25.289ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
18452024-12-20T21:46:25.289ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18462024-12-20T21:46:25.289ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18472024-12-20T21:46:25.289ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18482024-12-20T21:46:25.290ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18492024-12-20T21:46:25.292ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18502024-12-20T21:46:25.296ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
18512024-12-20T21:46:25.310ZINFOtest_mgs_metrics (InternalApi): No networking records needed for cockroach_db service
18522024-12-20T21:46:25.310ZINFOtest_mgs_metrics (InternalApi): No networking records needed for clickhouse service
18532024-12-20T21:46:25.311ZINFOtest_mgs_metrics (InternalApi): No networking records needed for internal_dns service
18542024-12-20T21:46:25.485ZINFOtest_mgs_metrics (InternalApi): Inserted networking records for external_dns service
18552024-12-20T21:46:25.601ZINFOtest_mgs_metrics (InternalApi): Inserted networking records for nexus service
18562024-12-20T21:46:25.601ZINFOtest_mgs_metrics (InternalApi): No networking records needed for crucible_pantry service
18572024-12-20T21:46:25.601ZINFOtest_mgs_metrics (InternalApi): Inserted service networking records
18582024-12-20T21:46:25.601ZINFOtest_mgs_metrics (InternalApi): Inserted physical disks
18592024-12-20T21:46:25.601ZINFOtest_mgs_metrics (InternalApi): Inserted zpools
18602024-12-20T21:46:25.601ZINFOtest_mgs_metrics (InternalApi): Inserted datasets
18612024-12-20T21:46:25.675ZINFOtest_mgs_metrics (InternalApi): Populated DNS tables for internal DNS
18622024-12-20T21:46:25.679ZINFOtest_mgs_metrics (InternalApi): Populated DNS tables for external DNS
18632024-12-20T21:46:25.679ZTRACtest_mgs_metrics (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
18642024-12-20T21:46:25.679ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18652024-12-20T21:46:25.679ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18662024-12-20T21:46:25.679ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18672024-12-20T21:46:25.680ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18682024-12-20T21:46:25.681ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18692024-12-20T21:46:25.685ZDEBGtest_mgs_metrics (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
18702024-12-20T21:46:25.685ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
18712024-12-20T21:46:25.685ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18722024-12-20T21:46:25.685ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18732024-12-20T21:46:25.685ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18742024-12-20T21:46:25.686ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18752024-12-20T21:46:25.690ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18762024-12-20T21:46:25.690ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
18772024-12-20T21:46:25.722ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
18782024-12-20T21:46:25.722ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18792024-12-20T21:46:25.722ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18802024-12-20T21:46:25.722ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18812024-12-20T21:46:25.723ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18822024-12-20T21:46:25.724ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18832024-12-20T21:46:25.729ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
18842024-12-20T21:46:25.729ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
18852024-12-20T21:46:25.729ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18862024-12-20T21:46:25.729ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18872024-12-20T21:46:25.729ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18882024-12-20T21:46:25.730ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18892024-12-20T21:46:25.733ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18902024-12-20T21:46:25.749ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
18912024-12-20T21:46:25.754ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
18922024-12-20T21:46:25.756ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18932024-12-20T21:46:25.756ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18942024-12-20T21:46:25.756ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
18952024-12-20T21:46:25.756ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18962024-12-20T21:46:25.762ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18972024-12-20T21:46:25.777ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
18982024-12-20T21:46:25.785ZINFOtest_mgs_metrics (InternalApi): Created recovery silo
18992024-12-20T21:46:25.791ZINFOtest_mgs_metrics (InternalApi): Created recovery user
19002024-12-20T21:46:25.829ZINFOtest_mgs_metrics (InternalApi): Created recovery user's password
19012024-12-20T21:46:25.829ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 2fc77303-3003-4d4c-9802-08a445a346d2, lookup_type: ById(2fc77303-3003-4d4c-9802-08a445a346d2) }
19022024-12-20T21:46:25.829ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 2fc77303-3003-4d4c-9802-08a445a346d2 resource_type = Silo
19032024-12-20T21:46:25.829ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19042024-12-20T21:46:25.830ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19052024-12-20T21:46:25.830ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19062024-12-20T21:46:25.836ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19072024-12-20T21:46:25.836ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19082024-12-20T21:46:25.836ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19092024-12-20T21:46:25.836ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19102024-12-20T21:46:25.843ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19112024-12-20T21:46:25.845ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 2fc77303-3003-4d4c-9802-08a445a346d2, lookup_type: ById(2fc77303-3003-4d4c-9802-08a445a346d2) } result = Ok(())
19122024-12-20T21:46:25.847ZDEBGtest_mgs_metrics (InternalApi): Generated role assignment queries
19132024-12-20T21:46:25.855ZINFOtest_mgs_metrics (InternalApi): Granted Silo privileges
19142024-12-20T21:46:25.856ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
19152024-12-20T21:46:25.856ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19162024-12-20T21:46:25.856ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19172024-12-20T21:46:25.856ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19182024-12-20T21:46:25.856ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19192024-12-20T21:46:25.860ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19202024-12-20T21:46:25.862ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
19212024-12-20T21:46:25.879ZINFOtest_mgs_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = f6868c12-530e-490c-b74a-04461fea5749
19222024-12-20T21:46:25.880ZDEBGtest_mgs_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = f6868c12-530e-490c-b74a-04461fea5749 time_started = 2024-12-20 21:46:23.167272 UTC
19232024-12-20T21:46:25.880ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 3.249576581s iteration = 3 status = {"collection_id":"f6868c12-530e-490c-b74a-04461fea5749","time_done":"2024-12-20 21:46:24.799691 UTC","time_started":"2024-12-20 21:46:23.167272 UTC"}
19242024-12-20T21:46:25.880ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
19252024-12-20T21:46:25.880ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 141.722\u{b5}s iteration = 2 status = {"error":"task disabled"}
19262024-12-20T21:46:25.893ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19272024-12-20T21:46:25.893ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19282024-12-20T21:46:25.894ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19292024-12-20T21:46:25.896ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19302024-12-20T21:46:25.896ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19312024-12-20T21:46:25.897ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19322024-12-20T21:46:25.901ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19332024-12-20T21:46:25.902ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19342024-12-20T21:46:25.903ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19352024-12-20T21:46:25.904ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
19362024-12-20T21:46:25.905ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
19372024-12-20T21:46:25.905ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19382024-12-20T21:46:25.905ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19392024-12-20T21:46:25.905ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19402024-12-20T21:46:25.909ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
19412024-12-20T21:46:25.918ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19422024-12-20T21:46:25.918ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19432024-12-20T21:46:25.918ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19442024-12-20T21:46:25.918ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19452024-12-20T21:46:25.918ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19462024-12-20T21:46:25.919ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19472024-12-20T21:46:25.919ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19482024-12-20T21:46:25.919ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19492024-12-20T21:46:25.920ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
19502024-12-20T21:46:25.920ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
19512024-12-20T21:46:25.920ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
19522024-12-20T21:46:25.920ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
19532024-12-20T21:46:25.920ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19542024-12-20T21:46:25.920ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19552024-12-20T21:46:25.920ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19562024-12-20T21:46:25.922ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
19572024-12-20T21:46:25.922ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19582024-12-20T21:46:25.922ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19592024-12-20T21:46:25.924ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19602024-12-20T21:46:25.932ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19612024-12-20T21:46:25.932ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19622024-12-20T21:46:25.932ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19632024-12-20T21:46:25.932ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19642024-12-20T21:46:25.932ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19652024-12-20T21:46:25.945ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
19662024-12-20T21:46:25.945ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19672024-12-20T21:46:25.945ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19682024-12-20T21:46:25.948ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19692024-12-20T21:46:25.951ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19702024-12-20T21:46:25.953ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19712024-12-20T21:46:25.953ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19722024-12-20T21:46:25.954ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19732024-12-20T21:46:25.955ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19742024-12-20T21:46:25.955ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19752024-12-20T21:46:25.956ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19762024-12-20T21:46:25.956ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19772024-12-20T21:46:25.957ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19782024-12-20T21:46:25.962ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
19792024-12-20T21:46:25.962ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
19802024-12-20T21:46:25.963ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19812024-12-20T21:46:25.963ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19822024-12-20T21:46:25.963ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19832024-12-20T21:46:25.965ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
19842024-12-20T21:46:25.966ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19852024-12-20T21:46:25.966ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19862024-12-20T21:46:25.966ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19872024-12-20T21:46:25.968ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19882024-12-20T21:46:25.969ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19892024-12-20T21:46:25.969ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19902024-12-20T21:46:25.969ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19912024-12-20T21:46:25.977ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19922024-12-20T21:46:25.985ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
19932024-12-20T21:46:25.985ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
19942024-12-20T21:46:25.985ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19952024-12-20T21:46:25.985ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19962024-12-20T21:46:25.985ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19972024-12-20T21:46:25.993ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19982024-12-20T21:46:25.993ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
19992024-12-20T21:46:25.994ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20002024-12-20T21:46:26.002ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20012024-12-20T21:46:26.002ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20022024-12-20T21:46:26.003ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20032024-12-20T21:46:26.008ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20042024-12-20T21:46:26.008ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20052024-12-20T21:46:26.009ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20062024-12-20T21:46:26.011ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
20072024-12-20T21:46:26.012ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20082024-12-20T21:46:26.012ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20092024-12-20T21:46:26.012ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20102024-12-20T21:46:26.012ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20112024-12-20T21:46:26.013ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
20122024-12-20T21:46:26.014ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20132024-12-20T21:46:26.014ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20142024-12-20T21:46:26.014ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20152024-12-20T21:46:26.015ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20162024-12-20T21:46:26.015ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20172024-12-20T21:46:26.015ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20182024-12-20T21:46:26.016ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20192024-12-20T21:46:26.017ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20202024-12-20T21:46:26.025ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
20212024-12-20T21:46:26.025ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
20222024-12-20T21:46:26.025ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
20232024-12-20T21:46:26.025ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20242024-12-20T21:46:26.025ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20252024-12-20T21:46:26.025ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20262024-12-20T21:46:26.026ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20272024-12-20T21:46:26.029ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
20282024-12-20T21:46:26.030ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20292024-12-20T21:46:26.030ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20302024-12-20T21:46:26.030ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20312024-12-20T21:46:26.032ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20322024-12-20T21:46:26.032ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20332024-12-20T21:46:26.032ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20342024-12-20T21:46:26.032ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20352024-12-20T21:46:26.034ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20362024-12-20T21:46:26.042ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
20372024-12-20T21:46:26.042ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20382024-12-20T21:46:26.042ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20392024-12-20T21:46:26.043ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20402024-12-20T21:46:26.047ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20412024-12-20T21:46:26.048ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20422024-12-20T21:46:26.048ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20432024-12-20T21:46:26.052ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20442024-12-20T21:46:26.052ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20452024-12-20T21:46:26.054ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20462024-12-20T21:46:26.055ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20472024-12-20T21:46:26.056ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20482024-12-20T21:46:26.056ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20492024-12-20T21:46:26.058ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20502024-12-20T21:46:26.059ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20512024-12-20T21:46:26.059ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20522024-12-20T21:46:26.061ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
20532024-12-20T21:46:26.062ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20542024-12-20T21:46:26.062ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20552024-12-20T21:46:26.062ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20562024-12-20T21:46:26.062ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20572024-12-20T21:46:26.065ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
20582024-12-20T21:46:26.065ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20592024-12-20T21:46:26.065ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20602024-12-20T21:46:26.065ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20612024-12-20T21:46:26.067ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20622024-12-20T21:46:26.067ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20632024-12-20T21:46:26.067ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20642024-12-20T21:46:26.067ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20652024-12-20T21:46:26.071ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20662024-12-20T21:46:26.080ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
20672024-12-20T21:46:26.081ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
20682024-12-20T21:46:26.081ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
20692024-12-20T21:46:26.081ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20702024-12-20T21:46:26.081ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20712024-12-20T21:46:26.081ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20722024-12-20T21:46:26.082ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20732024-12-20T21:46:26.083ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
20742024-12-20T21:46:26.083ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20752024-12-20T21:46:26.083ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20762024-12-20T21:46:26.084ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20772024-12-20T21:46:26.085ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20782024-12-20T21:46:26.086ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20792024-12-20T21:46:26.086ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20802024-12-20T21:46:26.086ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20812024-12-20T21:46:26.087ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20822024-12-20T21:46:26.096ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
20832024-12-20T21:46:26.096ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20842024-12-20T21:46:26.096ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20852024-12-20T21:46:26.097ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20862024-12-20T21:46:26.102ZDEBGtest_mgs_metrics (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 28aea37a-435d-4569-833a-6065c2eaea50, kind: Service { id: 67c5f200-c220-4da8-95cb-9522611a1492 }, name: Name("external-dns-67c5f200-c220-4da8-95cb-9522611a1492"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 0625a426-5680-4cd7-9477-d1da723fe2e1, kind: Service { id: cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176 }, name: Name("nexus-cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
20872024-12-20T21:46:26.103ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
20882024-12-20T21:46:26.103ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20892024-12-20T21:46:26.103ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20902024-12-20T21:46:26.103ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20912024-12-20T21:46:26.103ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20922024-12-20T21:46:26.105ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20932024-12-20T21:46:26.109ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
20942024-12-20T21:46:26.109ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20952024-12-20T21:46:26.109ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
20962024-12-20T21:46:26.109ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20972024-12-20T21:46:26.111ZTRACtest_mgs_metrics (nexus): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-12-20T21:23:45.899574Z, time_modified: 2024-12-20T21:46:25.862765Z, allowed_ips: None }
20982024-12-20T21:46:26.115ZDEBGtest_mgs_metrics (nexus): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
20992024-12-20T21:46:26.115ZDEBGtest_mgs_metrics (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 28aea37a-435d-4569-833a-6065c2eaea50, kind: Service { id: 67c5f200-c220-4da8-95cb-9522611a1492 }, name: Name("external-dns-67c5f200-c220-4da8-95cb-9522611a1492"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 0625a426-5680-4cd7-9477-d1da723fe2e1, kind: Service { id: cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176 }, name: Name("nexus-cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
21002024-12-20T21:46:26.115ZDEBGtest_mgs_metrics (nexus): resolved 2 rules for sleds
21012024-12-20T21:46:26.302ZDEBGtest_mgs_metrics (nexus): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2024-12-20T21:46:21.423841Z, time_modified: 2024-12-20T21:46:21.423841Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(34047), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
21022024-12-20T21:46:26.303ZDEBGtest_mgs_metrics (nexus): sending firewall rules to sled agents
21032024-12-20T21:46:26.303ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
21042024-12-20T21:46:26.303ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {} }
21052024-12-20T21:46:26.303ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
21062024-12-20T21:46:26.307ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
21072024-12-20T21:46:26.307ZTRACtest_mgs_metrics (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21082024-12-20T21:46:26.307ZTRACtest_mgs_metrics (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
21092024-12-20T21:46:26.307ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {} }
21102024-12-20T21:46:26.310ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
21112024-12-20T21:46:26.311ZDEBGtest_mgs_metrics (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
21122024-12-20T21:46:26.313ZDEBGtest_mgs_metrics (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
21132024-12-20T21:46:26.358ZDEBGtest_mgs_metrics (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:34047/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21142024-12-20T21:46:26.360ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34047 remote_addr = [::1]:41264 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
21152024-12-20T21:46:26.360ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41264 req_id = c6a0b724-bde9-40fd-b339-9c408b69beca sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21162024-12-20T21:46:26.360ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 627 local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41264 req_id = c6a0b724-bde9-40fd-b339-9c408b69beca response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21172024-12-20T21:46:26.361ZDEBGtest_mgs_metrics (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:34047/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "c6a0b724-bde9-40fd-b339-9c408b69beca", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
21182024-12-20T21:46:26.361ZINFOtest_mgs_metrics (nexus): updated firewall rules on 1 sleds
21192024-12-20T21:46:26.362ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
21202024-12-20T21:46:26.362ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {} }
21212024-12-20T21:46:26.363ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
21222024-12-20T21:46:26.363ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
21232024-12-20T21:46:26.363ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
21242024-12-20T21:46:26.363ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
21252024-12-20T21:46:26.363ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21262024-12-20T21:46:26.364ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21272024-12-20T21:46:26.364ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
21282024-12-20T21:46:26.365ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
21292024-12-20T21:46:26.365ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
21302024-12-20T21:46:26.365ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
21312024-12-20T21:46:26.365ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
21322024-12-20T21:46:26.365ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21332024-12-20T21:46:26.365ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21342024-12-20T21:46:26.365ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
21352024-12-20T21:46:26.365ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
21362024-12-20T21:46:26.365ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
21372024-12-20T21:46:26.365ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
21382024-12-20T21:46:26.365ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21392024-12-20T21:46:26.365ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
21402024-12-20T21:46:26.367ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21412024-12-20T21:46:26.369ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21422024-12-20T21:46:26.369ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Signaled
21432024-12-20T21:46:26.370ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
21442024-12-20T21:46:26.370ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21452024-12-20T21:46:26.370ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
21462024-12-20T21:46:26.370ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
21472024-12-20T21:46:26.372ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
21482024-12-20T21:46:26.372ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: 63 dc 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 c............_na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:61177 req_id = 6f2af95c-237d-4a54-86c6-5c7bc4b77b02
21492024-12-20T21:46:26.372ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25564,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61177 req_id = 6f2af95c-237d-4a54-86c6-5c7bc4b77b02
21502024-12-20T21:46:26.373ZDEBGtest_mgs_metrics (store): query key key = _nameservice._tcp
21512024-12-20T21:46:26.373ZDEBGtest_mgs_metrics (store): query key key = b83e9e5e-6a54-4940-bca6-01e138d611f0.host
21522024-12-20T21:46:26.373ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61177 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 34419, target: Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal") })) }] req_id = 6f2af95c-237d-4a54-86c6-5c7bc4b77b02
21532024-12-20T21:46:26.373ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 87 aa 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 ............._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59546 req_id = ec3cce9d-372b-4281-a770-b02e6085a9b7
21542024-12-20T21:46:26.374ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34730,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59546 req_id = ec3cce9d-372b-4281-a770-b02e6085a9b7
21552024-12-20T21:46:26.374ZDEBGtest_mgs_metrics (store): query key key = _external-dns._tcp
21562024-12-20T21:46:26.374ZDEBGtest_mgs_metrics (store): query key key = 67c5f200-c220-4da8-95cb-9522611a1492.host
21572024-12-20T21:46:26.374ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59546 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40731, target: Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal") })) }] req_id = ec3cce9d-372b-4281-a770-b02e6085a9b7
21582024-12-20T21:46:26.375ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 34419, target: Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2110, tv_nsec: 157996862 } })
21592024-12-20T21:46:26.376ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
21602024-12-20T21:46:26.376ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40731, target: Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2110, tv_nsec: 158357938 } })
21612024-12-20T21:46:26.377ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
21622024-12-20T21:46:26.377ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
21632024-12-20T21:46:26.378ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21642024-12-20T21:46:26.378ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
21652024-12-20T21:46:26.378ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21662024-12-20T21:46:26.378ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21672024-12-20T21:46:26.378ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
21682024-12-20T21:46:26.378ZTRACtest_mgs_metrics (ServiceBalancer): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21692024-12-20T21:46:26.378ZTRACtest_mgs_metrics (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
21702024-12-20T21:46:26.378ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {} }
21712024-12-20T21:46:26.386ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
21722024-12-20T21:46:26.386ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
21732024-12-20T21:46:26.386ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21742024-12-20T21:46:26.387ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
21752024-12-20T21:46:26.387ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21762024-12-20T21:46:26.388ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21772024-12-20T21:46:26.388ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: f8 cb 01 00 00 01 00 00 00 00 00 01 24 62 38 33 ............$b83\n0010: 65 39 65 35 65 2d 36 61 35 34 2d 34 39 34 30 2d e9e5e-6a54-4940-\n0020: 62 63 61 36 2d 30 31 65 31 33 38 64 36 31 31 66 bca6-01e138d611f\n0030: 30 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 0.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:59158 req_id = d95aa445-e78c-4fb8-9d40-6c8a1b638776
21782024-12-20T21:46:26.388ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63691,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 98,\n 56,\n 51,\n 101,\n 57,\n 101,\n 53,\n 101,\n 45,\n 54,\n 97,\n 53,\n 52,\n 45,\n 52,\n 57,\n 52,\n 48,\n 45,\n 98,\n 99,\n 97,\n 54,\n 45,\n 48,\n 49,\n 101,\n 49,\n 51,\n 56,\n 100,\n 54,\n 49,\n 49,\n 102,\n 48,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59158 req_id = d95aa445-e78c-4fb8-9d40-6c8a1b638776
21792024-12-20T21:46:26.388ZDEBGtest_mgs_metrics (store): query key key = b83e9e5e-6a54-4940-bca6-01e138d611f0.host
21802024-12-20T21:46:26.388ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59158 query = LowerQuery { name: LowerName(Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal.")), original: Query { name: Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("b83e9e5e-6a54-4940-bca6-01e138d611f0.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d95aa445-e78c-4fb8-9d40-6c8a1b638776
21812024-12-20T21:46:26.388ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: b2 ac 01 00 00 01 00 00 00 00 00 01 24 36 37 63 ............$67c\n0010: 35 66 32 30 30 2d 63 32 32 30 2d 34 64 61 38 2d 5f200-c220-4da8-\n0020: 39 35 63 62 2d 39 35 32 32 36 31 31 61 31 34 39 95cb-9522611a149\n0030: 32 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 2.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:54619 req_id = 283766dc-e4eb-4c5f-8af9-dd62491845e9
21822024-12-20T21:46:26.388ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 45740,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 54,\n 55,\n 99,\n 53,\n 102,\n 50,\n 48,\n 48,\n 45,\n 99,\n 50,\n 50,\n 48,\n 45,\n 52,\n 100,\n 97,\n 56,\n 45,\n 57,\n 53,\n 99,\n 98,\n 45,\n 57,\n 53,\n 50,\n 50,\n 54,\n 49,\n 49,\n 97,\n 49,\n 52,\n 57,\n 50,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54619 req_id = 283766dc-e4eb-4c5f-8af9-dd62491845e9
21832024-12-20T21:46:26.388ZDEBGtest_mgs_metrics (store): query key key = 67c5f200-c220-4da8-95cb-9522611a1492.host
21842024-12-20T21:46:26.388ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54619 query = LowerQuery { name: LowerName(Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal.")), original: Query { name: Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("67c5f200-c220-4da8-95cb-9522611a1492.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 283766dc-e4eb-4c5f-8af9-dd62491845e9
21852024-12-20T21:46:26.389ZINFOtest_mgs_metrics (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:34419] } background_task = dns_servers_internal dns_group = internal
21862024-12-20T21:46:26.395ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 25.399597ms iteration = 2 status = {"addresses":["[::1]:34419"]}
21872024-12-20T21:46:26.395ZINFOtest_mgs_metrics (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:40731] } background_task = dns_servers_external dns_group = external
21882024-12-20T21:46:26.395ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 24.685866ms iteration = 2 status = {"addresses":["[::1]:40731"]}
21892024-12-20T21:46:26.395ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
21902024-12-20T21:46:26.395ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config
21912024-12-20T21:46:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 26.18\u{b5}s iteration = 2 status = {"error":"no config"}
21922024-12-20T21:46:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
21932024-12-20T21:46:26.396ZWARNtest_mgs_metrics (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config
21942024-12-20T21:46:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 18.93\u{b5}s iteration = 2 status = {"error":"no config"}
21952024-12-20T21:46:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21962024-12-20T21:46:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21972024-12-20T21:46:26.396ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
21982024-12-20T21:46:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21992024-12-20T21:46:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22002024-12-20T21:46:26.396ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22012024-12-20T21:46:26.399ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
22022024-12-20T21:46:26.399ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
22032024-12-20T21:46:26.399ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
22042024-12-20T21:46:26.399ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
22052024-12-20T21:46:26.400ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22062024-12-20T21:46:26.406ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22072024-12-20T21:46:26.408ZDEBGtest_mgs_metrics (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22082024-12-20T21:46:26.414ZDEBGtest_mgs_metrics (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
22092024-12-20T21:46:26.414ZINFOtest_mgs_metrics (nexus): Rack initialized
22102024-12-20T21:46:26.415ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22112024-12-20T21:46:26.416ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22122024-12-20T21:46:26.416ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22132024-12-20T21:46:26.416ZDEBGtest_mgs_metrics (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
22142024-12-20T21:46:26.416ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
22152024-12-20T21:46:26.416ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
22162024-12-20T21:46:26.416ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
22172024-12-20T21:46:26.416ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22182024-12-20T21:46:26.416ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
22192024-12-20T21:46:26.416ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22202024-12-20T21:46:26.417ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22212024-12-20T21:46:26.417ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22222024-12-20T21:46:26.422ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22232024-12-20T21:46:26.424ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
22242024-12-20T21:46:26.424ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22252024-12-20T21:46:26.424ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
22262024-12-20T21:46:26.424ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22272024-12-20T21:46:26.425ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22282024-12-20T21:46:26.427ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22292024-12-20T21:46:26.433ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22302024-12-20T21:46:26.433ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
22312024-12-20T21:46:26.433ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22322024-12-20T21:46:26.434ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22332024-12-20T21:46:26.440ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22342024-12-20T21:46:26.447ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
22352024-12-20T21:46:26.449ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
22362024-12-20T21:46:26.449ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22372024-12-20T21:46:26.449ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22382024-12-20T21:46:26.449ZDEBGtest_mgs_metrics (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(f6868c12-530e-490c-b74a-04461fea5749, 0)]
22392024-12-20T21:46:26.449ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
22402024-12-20T21:46:26.449ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22412024-12-20T21:46:26.449ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22422024-12-20T21:46:26.450ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22432024-12-20T21:46:26.450ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 08 fa 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51542 req_id = b4a7a075-c8df-4e5e-9e8e-561a91ca6eb0
22442024-12-20T21:46:26.451ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2298,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51542 req_id = b4a7a075-c8df-4e5e-9e8e-561a91ca6eb0
22452024-12-20T21:46:26.451ZDEBGtest_mgs_metrics (store): query key key = _mgs._tcp
22462024-12-20T21:46:26.451ZDEBGtest_mgs_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
22472024-12-20T21:46:26.451ZDEBGtest_mgs_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
22482024-12-20T21:46:26.452ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51542 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36481, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 45883, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = b4a7a075-c8df-4e5e-9e8e-561a91ca6eb0
22492024-12-20T21:46:26.452ZDEBGtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 36481, target: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 45883, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2110, tv_nsec: 234862854 } })
22502024-12-20T21:46:26.452ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
22512024-12-20T21:46:26.452ZTRACtest_mgs_metrics (DnsResolver): lookup_service_targets: looking up SRV target
22522024-12-20T21:46:26.473ZDEBGtest_mgs_metrics (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
22532024-12-20T21:46:26.473ZDEBGtest_mgs_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = ead33721-f0bb-4bca-848e-7534bf0d54a0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22542024-12-20T21:46:26.473ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
22552024-12-20T21:46:26.474ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22562024-12-20T21:46:26.475ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
22572024-12-20T21:46:26.475ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22582024-12-20T21:46:26.480ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22592024-12-20T21:46:26.481ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 28 8a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e (...........-den\n0010: 64 72 69 74 65 2d 30 33 39 62 65 35 36 30 2d 35 drite-039be560-5\n0020: 34 63 63 2d 34 39 65 33 2d 38 38 64 66 2d 31 61 4cc-49e3-88df-1a\n0030: 32 39 64 61 64 62 66 39 31 33 04 68 6f 73 74 0d 29dadbf913.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55445 req_id = c2fdb839-8fc9-436c-8e5d-c9379cfbb7a7
22602024-12-20T21:46:26.481ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10378,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 48,\n 51,\n 57,\n 98,\n 101,\n 53,\n 54,\n 48,\n 45,\n 53,\n 52,\n 99,\n 99,\n 45,\n 52,\n 57,\n 101,\n 51,\n 45,\n 56,\n 56,\n 100,\n 102,\n 45,\n 49,\n 97,\n 50,\n 57,\n 100,\n 97,\n 100,\n 98,\n 102,\n 57,\n 49,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55445 req_id = c2fdb839-8fc9-436c-8e5d-c9379cfbb7a7
22612024-12-20T21:46:26.482ZDEBGtest_mgs_metrics (store): query key key = dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host
22622024-12-20T21:46:26.482ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55445 query = LowerQuery { name: LowerName(Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-039be560-54cc-49e3-88df-1a29dadbf913.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c2fdb839-8fc9-436c-8e5d-c9379cfbb7a7
22632024-12-20T21:46:26.482ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6a 73 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e js..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53004 req_id = 123c2652-c7e2-4c26-a704-f6ac5655319e
22642024-12-20T21:46:26.482ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27251,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53004 req_id = 123c2652-c7e2-4c26-a704-f6ac5655319e
22652024-12-20T21:46:26.482ZDEBGtest_mgs_metrics (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
22662024-12-20T21:46:26.482ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53004 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 123c2652-c7e2-4c26-a704-f6ac5655319e
22672024-12-20T21:46:26.645ZTRACtest_mgs_metrics (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
22682024-12-20T21:46:26.646ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22692024-12-20T21:46:26.660ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22702024-12-20T21:46:26.660ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
22712024-12-20T21:46:26.660ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22722024-12-20T21:46:26.664ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22732024-12-20T21:46:26.665ZDEBGtest_mgs_metrics (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
22742024-12-20T21:46:26.665ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
22752024-12-20T21:46:26.665ZDEBGtest_mgs_metrics (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
22762024-12-20T21:46:26.666ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
22772024-12-20T21:46:26.666ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22782024-12-20T21:46:26.666ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
22792024-12-20T21:46:26.666ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22802024-12-20T21:46:26.666ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22812024-12-20T21:46:26.666ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
22822024-12-20T21:46:26.666ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22832024-12-20T21:46:26.666ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
22842024-12-20T21:46:26.666ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22852024-12-20T21:46:26.668ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22862024-12-20T21:46:26.669ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22872024-12-20T21:46:26.669ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
22882024-12-20T21:46:26.670ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22892024-12-20T21:46:26.682ZDEBGtest_mgs_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = ead33721-f0bb-4bca-848e-7534bf0d54a0 dns_zone_name = control-plane.oxide.internal found_so_far = 18 version = 1
22902024-12-20T21:46:26.682ZDEBGtest_mgs_metrics (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22912024-12-20T21:46:26.682ZINFOtest_mgs_metrics (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal generation = 1
22922024-12-20T21:46:26.682ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 319.57687ms iteration = 2 status = {"generation":1}
22932024-12-20T21:46:26.682ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22942024-12-20T21:46:26.689ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22952024-12-20T21:46:26.690ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
22962024-12-20T21:46:26.690ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22972024-12-20T21:46:26.690ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22982024-12-20T21:46:26.690ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
22992024-12-20T21:46:26.729ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:34419 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:34419] } uri = http://[::1]:34419/config
23002024-12-20T21:46:26.729ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23012024-12-20T21:46:26.732ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
23022024-12-20T21:46:26.732ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
23032024-12-20T21:46:26.733ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23042024-12-20T21:46:26.733ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23052024-12-20T21:46:26.743ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:34419 remote_addr = [::1]:58410
23062024-12-20T21:46:26.743ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 27 17 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c '............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61490 req_id = f468d4f0-2c44-4790-b86d-0b1f402ff5d6
23072024-12-20T21:46:26.743ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10007,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61490 req_id = f468d4f0-2c44-4790-b86d-0b1f402ff5d6
23082024-12-20T21:46:26.744ZDEBGtest_mgs_metrics (store): query key key = _clickhouse-admin-keeper._tcp
23092024-12-20T21:46:26.744ZERROtest_mgs_metrics (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 10007, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61490 req_id = f468d4f0-2c44-4790-b86d-0b1f402ff5d6
23102024-12-20T21:46:26.745ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
23112024-12-20T21:46:26.745ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23122024-12-20T21:46:26.745ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23132024-12-20T21:46:26.745ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23142024-12-20T21:46:26.746ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23152024-12-20T21:46:26.747ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:34419 method = PUT remote_addr = [::1]:58410 req_id = 39536dd0-db29-4189-8491-4c9be2147d0f uri = /config
23162024-12-20T21:46:26.748ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection background_task = inventory_collection
23172024-12-20T21:46:26.748ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:36481
23182024-12-20T21:46:26.748ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/ignition
23192024-12-20T21:46:26.749ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 1 req_id = 39536dd0-db29-4189-8491-4c9be2147d0f
23202024-12-20T21:46:26.749ZINFOtest_mgs_metrics (store): updated generation new_generation = 1 req_id = 39536dd0-db29-4189-8491-4c9be2147d0f
23212024-12-20T21:46:26.749ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 1573 local_addr = [::1]:34419 method = PUT remote_addr = [::1]:58410 req_id = 39536dd0-db29-4189-8491-4c9be2147d0f response_code = 204 uri = /config
23222024-12-20T21:46:26.749ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:34419 generation = 1 result = Ok(Response { url: "http://[::1]:34419/config", status: 204, headers: {"x-request-id": "39536dd0-db29-4189-8491-4c9be2147d0f", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} }) servers = DnsServersList { addresses: [[::1]:34419] }
23232024-12-20T21:46:26.749ZINFOtest_mgs_metrics (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:34419 generation = 1 servers = DnsServersList { addresses: [[::1]:34419] }
23242024-12-20T21:46:26.750ZINFOtest_mgs_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:34419] }
23252024-12-20T21:46:26.750ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 60.415479ms iteration = 3 status = {"generation":1,"server_results":{"[::1]:34419":{"Ok":null}}}
23262024-12-20T21:46:26.753ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "43175aff-784b-4ee5-99ad-01b4d783f8e3", "content-length": "835", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
23272024-12-20T21:46:26.763ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0
23282024-12-20T21:46:26.763ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23292024-12-20T21:46:26.764ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23302024-12-20T21:46:26.764ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23312024-12-20T21:46:26.764ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23322024-12-20T21:46:26.766ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e98f55e6-0074-4768-924b-4240f4e58cf2", "content-length": "734", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
23332024-12-20T21:46:26.767ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/sp/caboose?firmware_slot=0
23342024-12-20T21:46:26.767ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23352024-12-20T21:46:26.767ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23362024-12-20T21:46:26.767ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23372024-12-20T21:46:26.767ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23382024-12-20T21:46:26.769ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "295162fb-cb2b-4dfa-8ab2-fb470f128d1f", "content-length": "111", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
23392024-12-20T21:46:26.769ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/sp/caboose?firmware_slot=1
23402024-12-20T21:46:26.771ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23412024-12-20T21:46:26.787ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
23422024-12-20T21:46:26.791ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
23432024-12-20T21:46:26.792ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
23442024-12-20T21:46:26.792ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23452024-12-20T21:46:26.792ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23462024-12-20T21:46:26.792ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23472024-12-20T21:46:26.792ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23482024-12-20T21:46:26.794ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d119cb4-130f-4ca7-8283-76d6a2814498", "content-length": "111", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
23492024-12-20T21:46:26.794ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/caboose?firmware_slot=0
23502024-12-20T21:46:26.797ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23512024-12-20T21:46:26.798ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23522024-12-20T21:46:26.798ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23532024-12-20T21:46:26.799ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23542024-12-20T21:46:26.805ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "933b3121-959b-4b27-84f4-459162965ade", "content-length": "167", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
23552024-12-20T21:46:26.805ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/caboose?firmware_slot=1
23562024-12-20T21:46:26.805ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23572024-12-20T21:46:26.806ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23582024-12-20T21:46:26.806ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23592024-12-20T21:46:26.806ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23602024-12-20T21:46:26.809ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23612024-12-20T21:46:26.828ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
23622024-12-20T21:46:26.828ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23632024-12-20T21:46:26.829ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23642024-12-20T21:46:26.829ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23652024-12-20T21:46:26.829ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "11e1def5-5383-4f25-8cb1-41ccab94c559", "content-length": "167", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
23662024-12-20T21:46:26.829ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/stage0/caboose?firmware_slot=0
23672024-12-20T21:46:26.845ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23682024-12-20T21:46:26.845ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23692024-12-20T21:46:26.846ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23702024-12-20T21:46:26.872ZDEBGtest_mgs_metrics (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
23712024-12-20T21:46:26.872ZDEBGtest_mgs_metrics (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 72e28229-9366-457f-b244-29741aadcdc3 dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
23722024-12-20T21:46:26.872ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
23732024-12-20T21:46:26.873ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23742024-12-20T21:46:26.874ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
23752024-12-20T21:46:26.874ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23762024-12-20T21:46:26.875ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23772024-12-20T21:46:26.879ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23782024-12-20T21:46:26.880ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23792024-12-20T21:46:26.880ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23802024-12-20T21:46:26.895ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23812024-12-20T21:46:26.901ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
23822024-12-20T21:46:26.901ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
23832024-12-20T21:46:26.901ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23842024-12-20T21:46:26.902ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23852024-12-20T21:46:26.902ZINFOtest_mgs_metrics (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-20T21:46:19.410754Z, time_modified: 2024-12-20T21:46:19.410754Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 2fc77303-3003-4d4c-9802-08a445a346d2, db_silo: Silo { identity: SiloIdentity { id: 2fc77303-3003-4d4c-9802-08a445a346d2, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-12-20T21:46:25.679072Z, time_modified: 2024-12-20T21:46:25.679072Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: 2fc77303-3003-4d4c-9802-08a445a346d2, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 2fc77303-3003-4d4c-9802-08a445a346d2, db_silo: Silo { identity: SiloIdentity { id: 2fc77303-3003-4d4c-9802-08a445a346d2, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2024-12-20T21:46:25.679072Z, time_modified: 2024-12-20T21:46:25.679072Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
23862024-12-20T21:46:26.903ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 537.894289ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"2fc77303-3003-4d4c-9802-08a445a346d2","tls_certs":[]}},"default_endpoint":{"silo_id":"2fc77303-3003-4d4c-9802-08a445a346d2","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo 2fc77303-3003-4d4c-9802-08a445a346d2 with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
23872024-12-20T21:46:26.906ZDEBGtest_mgs_metrics (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 72e28229-9366-457f-b244-29741aadcdc3 dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
23882024-12-20T21:46:26.906ZDEBGtest_mgs_metrics (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
23892024-12-20T21:46:26.906ZINFOtest_mgs_metrics (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external generation = 2
23902024-12-20T21:46:26.907ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 542.826739ms iteration = 2 status = {"generation":2}
23912024-12-20T21:46:26.907ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
23922024-12-20T21:46:26.971ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:40731 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:40731] } uri = http://[::1]:40731/config
23932024-12-20T21:46:26.972ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23942024-12-20T21:46:26.972ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
23952024-12-20T21:46:26.973ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23962024-12-20T21:46:26.974ZINFOtest_mgs_metrics (http): accepted connection kind = dns local_addr = [::1]:40731 remote_addr = [::1]:51840
23972024-12-20T21:46:26.974ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6033d6d-509d-4bc5-aef9-99a2a99a5124", "content-length": "175", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
23982024-12-20T21:46:26.975ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/stage0/caboose?firmware_slot=1
23992024-12-20T21:46:26.979ZTRACtest_mgs_metrics (http): incoming request kind = dns local_addr = [::1]:40731 method = PUT remote_addr = [::1]:51840 req_id = 638aca9c-d696-48d0-a8dd-8cd430dd0043 uri = /config
24002024-12-20T21:46:26.984ZINFOtest_mgs_metrics (store): attempting generation update new_generation = 2 req_id = 638aca9c-d696-48d0-a8dd-8cd430dd0043
24012024-12-20T21:46:26.984ZINFOtest_mgs_metrics (store): pruning trees for generations newer than 0
24022024-12-20T21:46:26.984ZDEBGtest_mgs_metrics (store): creating tree tree_name = generation_2_zone_oxide-dev.test
24032024-12-20T21:46:26.985ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
24042024-12-20T21:46:26.985ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
24052024-12-20T21:46:26.985ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24062024-12-20T21:46:26.985ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24072024-12-20T21:46:26.986ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24082024-12-20T21:46:26.987ZDEBGtest_mgs_metrics (store): updating current config
24092024-12-20T21:46:26.988ZDEBGtest_mgs_metrics (store): flushing default tree
24102024-12-20T21:46:27.016ZINFOtest_mgs_metrics (store): pruning trees for generations older than 2 keep = 3
24112024-12-20T21:46:27.016ZINFOtest_mgs_metrics (store): updated generation new_generation = 2 req_id = 638aca9c-d696-48d0-a8dd-8cd430dd0043
24122024-12-20T21:46:27.017ZINFOtest_mgs_metrics (http): request completed kind = dns latency_us = 37305 local_addr = [::1]:40731 method = PUT remote_addr = [::1]:51840 req_id = 638aca9c-d696-48d0-a8dd-8cd430dd0043 response_code = 204 uri = /config
24132024-12-20T21:46:27.017ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
24142024-12-20T21:46:27.017ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24152024-12-20T21:46:27.018ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24162024-12-20T21:46:27.018ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24172024-12-20T21:46:27.033ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:40731 generation = 2 result = Ok(Response { url: "http://[::1]:40731/config", status: 204, headers: {"x-request-id": "638aca9c-d696-48d0-a8dd-8cd430dd0043", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} }) servers = DnsServersList { addresses: [[::1]:40731] }
24182024-12-20T21:46:27.034ZINFOtest_mgs_metrics (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:40731 generation = 2 servers = DnsServersList { addresses: [[::1]:40731] }
24192024-12-20T21:46:27.035ZINFOtest_mgs_metrics (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:40731] }
24202024-12-20T21:46:27.036ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 129.603727ms iteration = 3 status = {"generation":2,"server_results":{"[::1]:40731":{"Ok":null}}}
24212024-12-20T21:46:27.052ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24222024-12-20T21:46:27.052ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24232024-12-20T21:46:27.053ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24242024-12-20T21:46:27.053ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24252024-12-20T21:46:27.074ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24262024-12-20T21:46:27.118ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
24272024-12-20T21:46:27.118ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
24282024-12-20T21:46:27.118ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24292024-12-20T21:46:27.119ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24302024-12-20T21:46:27.119ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24312024-12-20T21:46:27.132ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24322024-12-20T21:46:27.133ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24332024-12-20T21:46:27.133ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24342024-12-20T21:46:27.136ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "008ca15c-44b9-4b09-816e-27c95ac487e0", "content-length": "175", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24352024-12-20T21:46:27.137ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/cmpa
24362024-12-20T21:46:27.137ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24372024-12-20T21:46:27.137ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24382024-12-20T21:46:27.138ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24392024-12-20T21:46:27.140ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f1a1198-f306-4a2a-8d3b-0ba614d65bc1", "content-length": "702", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24402024-12-20T21:46:27.140ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/cfpa
24412024-12-20T21:46:27.141ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24422024-12-20T21:46:27.141ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24432024-12-20T21:46:27.142ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24442024-12-20T21:46:27.151ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
24452024-12-20T21:46:27.151ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
24462024-12-20T21:46:27.152ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24472024-12-20T21:46:27.152ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24482024-12-20T21:46:27.153ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24492024-12-20T21:46:27.154ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21cef7c8-024d-4045-930d-7311421fddef", "content-length": "727", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24502024-12-20T21:46:27.154ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/cfpa
24512024-12-20T21:46:27.156ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6943a1ab-6633-4046-8d68-33bd8659800c", "content-length": "729", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24522024-12-20T21:46:27.156ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/0/component/rot/cfpa
24532024-12-20T21:46:27.158ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3c3a14d-f7cc-4f04-9b4a-64b741955fb9", "content-length": "728", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24542024-12-20T21:46:27.158ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1
24552024-12-20T21:46:27.160ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
24562024-12-20T21:46:27.163ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24572024-12-20T21:46:27.163ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24582024-12-20T21:46:27.163ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24592024-12-20T21:46:27.163ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "58cebc45-086e-4e1f-bd23-57159f880cbf", "content-length": "734", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24602024-12-20T21:46:27.163ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/sp/caboose?firmware_slot=0
24612024-12-20T21:46:27.163ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24622024-12-20T21:46:27.163ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24632024-12-20T21:46:27.164ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24642024-12-20T21:46:27.164ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24652024-12-20T21:46:27.167ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24662024-12-20T21:46:27.177ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
24672024-12-20T21:46:27.177ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
24682024-12-20T21:46:27.177ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
24692024-12-20T21:46:27.177ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
24702024-12-20T21:46:27.177ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24712024-12-20T21:46:27.177ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24722024-12-20T21:46:27.178ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24732024-12-20T21:46:27.182ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
24742024-12-20T21:46:27.182ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24752024-12-20T21:46:27.182ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24762024-12-20T21:46:27.183ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24772024-12-20T21:46:27.185ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
24782024-12-20T21:46:27.185ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24792024-12-20T21:46:27.185ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24802024-12-20T21:46:27.199ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24812024-12-20T21:46:27.203ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24822024-12-20T21:46:27.213ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
24832024-12-20T21:46:27.213ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24842024-12-20T21:46:27.213ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24852024-12-20T21:46:27.214ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24862024-12-20T21:46:27.214ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0ed705e-b889-4e67-a045-0da8268007b0", "content-length": "111", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24872024-12-20T21:46:27.215ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/sp/caboose?firmware_slot=1
24882024-12-20T21:46:27.217ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e79012d-266e-44e7-929d-fde46cb10a4f", "content-length": "111", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24892024-12-20T21:46:27.217ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/caboose?firmware_slot=0
24902024-12-20T21:46:27.219ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24912024-12-20T21:46:27.219ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24922024-12-20T21:46:27.219ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24932024-12-20T21:46:27.221ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e37226c1-b704-41a4-a930-12d54a17328e", "content-length": "167", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24942024-12-20T21:46:27.221ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/caboose?firmware_slot=1
24952024-12-20T21:46:27.223ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
24962024-12-20T21:46:27.223ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
24972024-12-20T21:46:27.223ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
24982024-12-20T21:46:27.225ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f897842e-556a-4caa-b62d-73aa74de03df", "content-length": "167", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
24992024-12-20T21:46:27.226ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/stage0/caboose?firmware_slot=0
25002024-12-20T21:46:27.226ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25012024-12-20T21:46:27.226ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25022024-12-20T21:46:27.227ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25032024-12-20T21:46:27.229ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25042024-12-20T21:46:27.229ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25052024-12-20T21:46:27.229ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25062024-12-20T21:46:27.232ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
25072024-12-20T21:46:27.232ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
25082024-12-20T21:46:27.232ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25092024-12-20T21:46:27.232ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25102024-12-20T21:46:27.233ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25112024-12-20T21:46:27.235ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
25122024-12-20T21:46:27.235ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25132024-12-20T21:46:27.235ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25142024-12-20T21:46:27.236ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25152024-12-20T21:46:27.237ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2ad4eb1-34ec-4348-85b5-cdf02f4307fb", "content-length": "175", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25162024-12-20T21:46:27.238ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/stage0/caboose?firmware_slot=1
25172024-12-20T21:46:27.238ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
25182024-12-20T21:46:27.238ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25192024-12-20T21:46:27.238ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25202024-12-20T21:46:27.239ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25212024-12-20T21:46:27.241ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25222024-12-20T21:46:27.250ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
25232024-12-20T21:46:27.250ZTRACtest_mgs_metrics (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
25242024-12-20T21:46:27.250ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
25252024-12-20T21:46:27.250ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
25262024-12-20T21:46:27.250ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25272024-12-20T21:46:27.250ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25282024-12-20T21:46:27.251ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25292024-12-20T21:46:27.256ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
25302024-12-20T21:46:27.256ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25312024-12-20T21:46:27.256ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25322024-12-20T21:46:27.260ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25332024-12-20T21:46:27.263ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
25342024-12-20T21:46:27.263ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25352024-12-20T21:46:27.263ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25362024-12-20T21:46:27.265ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25372024-12-20T21:46:27.268ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cfb9fb3-3378-4898-8d16-f5bcd28426c3", "content-length": "175", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25382024-12-20T21:46:27.282ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/cmpa
25392024-12-20T21:46:27.283ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25402024-12-20T21:46:27.291ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
25412024-12-20T21:46:27.292ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25422024-12-20T21:46:27.292ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25432024-12-20T21:46:27.292ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25442024-12-20T21:46:27.297ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "da2d5dae-47e9-418e-bbbc-4963acda3fe0", "content-length": "702", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25452024-12-20T21:46:27.297ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/cfpa
25462024-12-20T21:46:27.298ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99a8b91b-d32d-4be7-9de2-99b7ee776149", "content-length": "727", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25472024-12-20T21:46:27.299ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/cfpa
25482024-12-20T21:46:27.299ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f198851-8eba-4b94-9eeb-19fad17921f1", "content-length": "729", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25492024-12-20T21:46:27.300ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/switch/1/component/rot/cfpa
25502024-12-20T21:46:27.301ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "64c483c9-9f2c-4d6a-b0a1-05c2f7b23f08", "content-length": "728", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25512024-12-20T21:46:27.301ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0
25522024-12-20T21:46:27.302ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea72125c-03de-4aa2-bd88-841a5e0ec2d9", "content-length": "723", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25532024-12-20T21:46:27.303ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/sp/caboose?firmware_slot=0
25542024-12-20T21:46:27.305ZDEBGtest_mgs_metrics (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 28aea37a-435d-4569-833a-6065c2eaea50, kind: Service { id: 67c5f200-c220-4da8-95cb-9522611a1492 }, name: Name("external-dns-67c5f200-c220-4da8-95cb-9522611a1492"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 0625a426-5680-4cd7-9477-d1da723fe2e1, kind: Service { id: cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176 }, name: Name("nexus-cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
25552024-12-20T21:46:27.306ZTRACtest_mgs_metrics (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
25562024-12-20T21:46:27.306ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
25572024-12-20T21:46:27.306ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25582024-12-20T21:46:27.306ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25592024-12-20T21:46:27.306ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25602024-12-20T21:46:27.308ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25612024-12-20T21:46:27.312ZDEBGtest_mgs_metrics (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
25622024-12-20T21:46:27.312ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
25632024-12-20T21:46:27.312ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
25642024-12-20T21:46:27.312ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
25652024-12-20T21:46:27.315ZTRACtest_mgs_metrics (InternalApi): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2024-12-20T21:23:45.899574Z, time_modified: 2024-12-20T21:46:25.862765Z, allowed_ips: None }
25662024-12-20T21:46:27.315ZDEBGtest_mgs_metrics (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
25672024-12-20T21:46:27.315ZDEBGtest_mgs_metrics (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 28aea37a-435d-4569-833a-6065c2eaea50, kind: Service { id: 67c5f200-c220-4da8-95cb-9522611a1492 }, name: Name("external-dns-67c5f200-c220-4da8-95cb-9522611a1492"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 0625a426-5680-4cd7-9477-d1da723fe2e1, kind: Service { id: cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176 }, name: Name("nexus-cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
25682024-12-20T21:46:27.315ZDEBGtest_mgs_metrics (InternalApi): resolved 2 rules for sleds
25692024-12-20T21:46:27.316ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "614e4bd5-b13e-48e7-b970-db9d4281ec04", "content-length": "109", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25702024-12-20T21:46:27.316ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/sp/caboose?firmware_slot=1
25712024-12-20T21:46:27.320ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b88f223-6859-496b-aa15-1de8cde1b9b0", "content-length": "109", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25722024-12-20T21:46:27.320ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/caboose?firmware_slot=0
25732024-12-20T21:46:27.322ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bd2d150-131e-45f0-aa3b-84e1caf3cfcd", "content-length": "169", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25742024-12-20T21:46:27.322ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/caboose?firmware_slot=1
25752024-12-20T21:46:27.324ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bb9e4c6-845f-4ddf-9aba-c85f0d9e2a08", "content-length": "169", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25762024-12-20T21:46:27.335ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/stage0/caboose?firmware_slot=0
25772024-12-20T21:46:27.335ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7051dd65-65e9-4f0e-a207-4c734d14846d", "content-length": "178", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25782024-12-20T21:46:27.335ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/stage0/caboose?firmware_slot=1
25792024-12-20T21:46:27.335ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ccfa287-f1b6-4d92-90cf-93feb2dc94dd", "content-length": "178", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25802024-12-20T21:46:27.336ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/cmpa
25812024-12-20T21:46:27.336ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75a7182f-8e94-41a2-b7b1-e4fb53684b48", "content-length": "702", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25822024-12-20T21:46:27.336ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/cfpa
25832024-12-20T21:46:27.336ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9db87fb5-e0c9-473e-954f-b8a087512462", "content-length": "727", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25842024-12-20T21:46:27.336ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/cfpa
25852024-12-20T21:46:27.336ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33ee7625-203e-47c0-adcc-dbbc92bed481", "content-length": "729", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25862024-12-20T21:46:27.336ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/0/component/rot/cfpa
25872024-12-20T21:46:27.338ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cefd71c-0ae8-42aa-97b4-e073b68b8650", "content-length": "728", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25882024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1
25892024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a76d9f14-9e0d-4ab4-b932-55836bb6b3d6", "content-length": "723", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25902024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/sp/caboose?firmware_slot=0
25912024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f243702-6fba-4531-85ae-dc5679af334b", "content-length": "109", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25922024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/sp/caboose?firmware_slot=1
25932024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26121305-b382-408e-baa5-d3a8bdf425e7", "content-length": "109", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25942024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/caboose?firmware_slot=0
25952024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5173ce46-3851-4bcd-ac90-197674d471fc", "content-length": "169", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25962024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/caboose?firmware_slot=1
25972024-12-20T21:46:27.340ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ef6cf7e-9e43-4f7c-9d43-ca08293d2913", "content-length": "169", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
25982024-12-20T21:46:27.341ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/stage0/caboose?firmware_slot=0
25992024-12-20T21:46:27.342ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d255578a-b133-4545-b640-92a28fa233ad", "content-length": "178", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
26002024-12-20T21:46:27.344ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/stage0/caboose?firmware_slot=1
26012024-12-20T21:46:27.344ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84a7d447-85e6-43f6-92ef-51192cc4321c", "content-length": "178", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
26022024-12-20T21:46:27.344ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/cmpa
26032024-12-20T21:46:27.345ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a61312fa-f3f3-4131-a86b-23dd8f3f628c", "content-length": "702", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
26042024-12-20T21:46:27.346ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/cfpa
26052024-12-20T21:46:27.346ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ef5c11a-536a-4592-afa8-a7752ff5fd56", "content-length": "727", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
26062024-12-20T21:46:27.346ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/cfpa
26072024-12-20T21:46:27.347ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffefa471-12b3-429a-ae75-7868376ac72d", "content-length": "729", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
26082024-12-20T21:46:27.349ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:36481 method = GET uri = http://[::1]:36481/sp/sled/1/component/rot/cfpa
26092024-12-20T21:46:27.350ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:36481 result = Ok(Response { url: "http://[::1]:36481/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4107d7b-f6f5-49e1-ae98-318581e51c7e", "content-length": "728", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
26102024-12-20T21:46:27.350ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:45883
26112024-12-20T21:46:27.350ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/ignition
26122024-12-20T21:46:27.352ZDEBGtest_mgs_metrics (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2024-12-20T21:46:21.423841Z, time_modified: 2024-12-20T21:46:21.423841Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(34047), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
26132024-12-20T21:46:27.352ZDEBGtest_mgs_metrics (InternalApi): sending firewall rules to sled agents
26142024-12-20T21:46:27.352ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
26152024-12-20T21:46:27.352ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
26162024-12-20T21:46:27.354ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
26172024-12-20T21:46:27.354ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0922f889-2112-4489-8a0e-1a7621e20002", "content-length": "835", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
26182024-12-20T21:46:27.354ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/sp/switch/0
26192024-12-20T21:46:27.356ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f00d3bd-eb25-42d4-92e3-3325b21e02ba", "content-length": "734", "date": "Fri, 20 Dec 2024 21:46:26 GMT"} })
26202024-12-20T21:46:27.357ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/sp/switch/1
26212024-12-20T21:46:27.357ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
26222024-12-20T21:46:27.357ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
26232024-12-20T21:46:27.357ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
26242024-12-20T21:46:27.357ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
26252024-12-20T21:46:27.358ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
26262024-12-20T21:46:27.359ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a575b1f8-cf21-4d08-8510-c96aef4cd22a", "content-length": "734", "date": "Fri, 20 Dec 2024 21:46:27 GMT"} })
26272024-12-20T21:46:27.360ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/sp/sled/0
26282024-12-20T21:46:27.360ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26292024-12-20T21:46:27.365ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
26302024-12-20T21:46:27.403ZDEBGtest_mgs_metrics (InternalApi): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:34047/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
26312024-12-20T21:46:27.404ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34047 remote_addr = [::1]:41268 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
26322024-12-20T21:46:27.406ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41268 req_id = 19b86c51-b5e1-4766-91a3-3e181b7acc4f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
26332024-12-20T21:46:27.407ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c98a5a6-e593-4489-b35f-a1d613d2cd1d", "content-length": "723", "date": "Fri, 20 Dec 2024 21:46:27 GMT"} })
26342024-12-20T21:46:27.407ZDEBGtest_mgs_metrics (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:45883 method = GET uri = http://[::1]:45883/sp/sled/1
26352024-12-20T21:46:27.408ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 1567 local_addr = [::1]:34047 method = PUT remote_addr = [::1]:41268 req_id = 19b86c51-b5e1-4766-91a3-3e181b7acc4f response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
26362024-12-20T21:46:27.408ZDEBGtest_mgs_metrics (InternalApi): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:34047/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "19b86c51-b5e1-4766-91a3-3e181b7acc4f", "date": "Fri, 20 Dec 2024 21:46:27 GMT"} })
26372024-12-20T21:46:27.409ZINFOtest_mgs_metrics (InternalApi): updated firewall rules on 1 sleds
26382024-12-20T21:46:27.409ZINFOtest_mgs_metrics (nexus): plumbed initial IP allowlist
26392024-12-20T21:46:27.454ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = / versions = all
26402024-12-20T21:46:27.454ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /access versions = all
26412024-12-20T21:46:27.454ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /assets/{path} versions = all
26422024-12-20T21:46:27.454ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /device/auth versions = all
26432024-12-20T21:46:27.455ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /device/confirm versions = all
26442024-12-20T21:46:27.455ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /device/success versions = all
26452024-12-20T21:46:27.455ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /device/token versions = all
26462024-12-20T21:46:27.455ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /device/verify versions = all
26472024-12-20T21:46:27.455ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /experimental/v1/probes versions = all
26482024-12-20T21:46:27.455ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /experimental/v1/probes versions = all
26492024-12-20T21:46:27.455ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /experimental/v1/probes/{probe} versions = all
26502024-12-20T21:46:27.456ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /experimental/v1/probes/{probe} versions = all
26512024-12-20T21:46:27.456ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /images versions = all
26522024-12-20T21:46:27.456ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /login versions = all
26532024-12-20T21:46:27.456ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /login/{silo_name}/local versions = all
26542024-12-20T21:46:27.456ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
26552024-12-20T21:46:27.457ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
26562024-12-20T21:46:27.457ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
26572024-12-20T21:46:27.457ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /lookup/{path} versions = all
26582024-12-20T21:46:27.457ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /projects/{path} versions = all
26592024-12-20T21:46:27.458ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /projects-new versions = all
26602024-12-20T21:46:27.458ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /settings/{path} versions = all
26612024-12-20T21:46:27.458ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /system/{path} versions = all
26622024-12-20T21:46:27.458ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /utilization versions = all
26632024-12-20T21:46:27.458ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/certificates versions = all
26642024-12-20T21:46:27.458ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/certificates versions = all
26652024-12-20T21:46:27.458ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/certificates/{certificate} versions = all
26662024-12-20T21:46:27.458ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/certificates/{certificate} versions = all
26672024-12-20T21:46:27.460ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/disks versions = all
26682024-12-20T21:46:27.460ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/disks versions = all
26692024-12-20T21:46:27.460ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/disks/{disk} versions = all
26702024-12-20T21:46:27.460ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/disks/{disk} versions = all
26712024-12-20T21:46:27.460ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/disks/{disk}/bulk-write versions = all
26722024-12-20T21:46:27.460ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
26732024-12-20T21:46:27.460ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
26742024-12-20T21:46:27.460ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/disks/{disk}/finalize versions = all
26752024-12-20T21:46:27.461ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
26762024-12-20T21:46:27.461ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/floating-ips versions = all
26772024-12-20T21:46:27.461ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/floating-ips versions = all
26782024-12-20T21:46:27.461ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
26792024-12-20T21:46:27.462ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/floating-ips/{floating_ip} versions = all
26802024-12-20T21:46:27.462ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
26812024-12-20T21:46:27.462ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
26822024-12-20T21:46:27.463ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
26832024-12-20T21:46:27.463ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/groups versions = all
26842024-12-20T21:46:27.463ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/groups/{group_id} versions = all
26852024-12-20T21:46:27.463ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/images versions = all
26862024-12-20T21:46:27.463ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/images versions = all
26872024-12-20T21:46:27.465ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/images/{image} versions = all
26882024-12-20T21:46:27.465ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/images/{image} versions = all
26892024-12-20T21:46:27.465ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/images/{image}/demote versions = all
26902024-12-20T21:46:27.468ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/images/{image}/promote versions = all
26912024-12-20T21:46:27.468ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/instances versions = all
26922024-12-20T21:46:27.468ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/instances versions = all
26932024-12-20T21:46:27.468ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/instances/{instance} versions = all
26942024-12-20T21:46:27.469ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/instances/{instance} versions = all
26952024-12-20T21:46:27.469ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/instances/{instance} versions = all
26962024-12-20T21:46:27.469ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/instances/{instance}/disks versions = all
26972024-12-20T21:46:27.469ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/instances/{instance}/disks/attach versions = all
26982024-12-20T21:46:27.469ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/instances/{instance}/disks/detach versions = all
26992024-12-20T21:46:27.470ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/instances/{instance}/external-ips versions = all
27002024-12-20T21:46:27.470ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27012024-12-20T21:46:27.470ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27022024-12-20T21:46:27.470ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/instances/{instance}/reboot versions = all
27032024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/instances/{instance}/serial-console versions = all
27042024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
27052024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
27062024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/instances/{instance}/start versions = all
27072024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/instances/{instance}/stop versions = all
27082024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/internet-gateway-ip-addresses versions = all
27092024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/internet-gateway-ip-addresses versions = all
27102024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
27112024-12-20T21:46:27.471ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/internet-gateway-ip-pools versions = all
27122024-12-20T21:46:27.472ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/internet-gateway-ip-pools versions = all
27132024-12-20T21:46:27.472ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
27142024-12-20T21:46:27.472ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/internet-gateways versions = all
27152024-12-20T21:46:27.472ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/internet-gateways versions = all
27162024-12-20T21:46:27.472ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
27172024-12-20T21:46:27.472ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/internet-gateways/{gateway} versions = all
27182024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/ip-pools versions = all
27192024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/ip-pools/{pool} versions = all
27202024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/login/{silo_name}/local versions = all
27212024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/logout versions = all
27222024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/me versions = all
27232024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/me/groups versions = all
27242024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/me/ssh-keys versions = all
27252024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/me/ssh-keys versions = all
27262024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
27272024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
27282024-12-20T21:46:27.473ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/metrics/{metric_name} versions = all
27292024-12-20T21:46:27.474ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/network-interfaces versions = all
27302024-12-20T21:46:27.474ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/network-interfaces versions = all
27312024-12-20T21:46:27.474ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/network-interfaces/{interface} versions = all
27322024-12-20T21:46:27.474ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/network-interfaces/{interface} versions = all
27332024-12-20T21:46:27.474ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/network-interfaces/{interface} versions = all
27342024-12-20T21:46:27.474ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/ping versions = all
27352024-12-20T21:46:27.474ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/policy versions = all
27362024-12-20T21:46:27.474ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/policy versions = all
27372024-12-20T21:46:27.475ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/projects versions = all
27382024-12-20T21:46:27.475ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/projects versions = all
27392024-12-20T21:46:27.475ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/projects/{project} versions = all
27402024-12-20T21:46:27.475ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/projects/{project} versions = all
27412024-12-20T21:46:27.475ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/projects/{project} versions = all
27422024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/projects/{project}/policy versions = all
27432024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/projects/{project}/policy versions = all
27442024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/snapshots versions = all
27452024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/snapshots versions = all
27462024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/snapshots/{snapshot} versions = all
27472024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/snapshots/{snapshot} versions = all
27482024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/disks versions = all
27492024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
27502024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/racks versions = all
27512024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
27522024-12-20T21:46:27.476ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/sleds versions = all
27532024-12-20T21:46:27.477ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/hardware/sleds versions = all
27542024-12-20T21:46:27.477ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
27552024-12-20T21:46:27.477ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
27562024-12-20T21:46:27.477ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
27572024-12-20T21:46:27.477ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
27582024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
27592024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/switch-port versions = all
27602024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
27612024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
27622024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
27632024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/switches versions = all
27642024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
27652024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/identity-providers versions = all
27662024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/identity-providers/local/users versions = all
27672024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
27682024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
27692024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/identity-providers/saml versions = all
27702024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
27712024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/ip-pools versions = all
27722024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/ip-pools versions = all
27732024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
27742024-12-20T21:46:27.478ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/ip-pools/{pool} versions = all
27752024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/ip-pools/{pool} versions = all
27762024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
27772024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
27782024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
27792024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
27802024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
27812024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
27822024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
27832024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
27842024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/ip-pools-service versions = all
27852024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/ip-pools-service/ranges versions = all
27862024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
27872024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
27882024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/metrics/{metric_name} versions = all
27892024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/address-lot versions = all
27902024-12-20T21:46:27.479ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/networking/address-lot versions = all
27912024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
27922024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
27932024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/allow-list versions = all
27942024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/networking/allow-list versions = all
27952024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/networking/bfd-disable versions = all
27962024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/networking/bfd-enable versions = all
27972024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/bfd-status versions = all
27982024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/networking/bgp versions = all
27992024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/bgp versions = all
28002024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/networking/bgp versions = all
28012024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/bgp-announce-set versions = all
28022024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
28032024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
28042024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
28052024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/bgp-exported versions = all
28062024-12-20T21:46:27.480ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/bgp-message-history versions = all
28072024-12-20T21:46:27.481ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
28082024-12-20T21:46:27.481ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/bgp-status versions = all
28092024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/loopback-address versions = all
28102024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/networking/loopback-address versions = all
28112024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
28122024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
28132024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/switch-port-settings versions = all
28142024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/networking/switch-port-settings versions = all
28152024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
28162024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/policy versions = all
28172024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/policy versions = all
28182024-12-20T21:46:27.482ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/roles versions = all
28192024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/roles/{role_name} versions = all
28202024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/silo-quotas versions = all
28212024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/silos versions = all
28222024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/silos versions = all
28232024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/system/silos/{silo} versions = all
28242024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/silos/{silo} versions = all
28252024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
28262024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/silos/{silo}/policy versions = all
28272024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/silos/{silo}/policy versions = all
28282024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/silos/{silo}/quotas versions = all
28292024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
28302024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/system/timeseries/query versions = all
28312024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/timeseries/schemas versions = all
28322024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/system/update/repository versions = all
28332024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/update/repository/{system_version} versions = all
28342024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/users versions = all
28352024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/users/{user_id} versions = all
28362024-12-20T21:46:27.483ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/users-builtin versions = all
28372024-12-20T21:46:27.484ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/users-builtin/{user} versions = all
28382024-12-20T21:46:27.484ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/utilization/silos versions = all
28392024-12-20T21:46:27.484ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/system/utilization/silos/{silo} versions = all
28402024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/timeseries/query versions = all
28412024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/users versions = all
28422024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/utilization versions = all
28432024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpc-firewall-rules versions = all
28442024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/vpc-firewall-rules versions = all
28452024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpc-router-routes versions = all
28462024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/vpc-router-routes versions = all
28472024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
28482024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpc-router-routes/{route} versions = all
28492024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/vpc-router-routes/{route} versions = all
28502024-12-20T21:46:27.485ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpc-routers versions = all
28512024-12-20T21:46:27.486ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/vpc-routers versions = all
28522024-12-20T21:46:27.486ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/vpc-routers/{router} versions = all
28532024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpc-routers/{router} versions = all
28542024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/vpc-routers/{router} versions = all
28552024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpc-subnets versions = all
28562024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/vpc-subnets versions = all
28572024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
28582024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpc-subnets/{subnet} versions = all
28592024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
28602024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
28612024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpcs versions = all
28622024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = POST path = /v1/vpcs versions = all
28632024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = DELETE path = /v1/vpcs/{vpc} versions = all
28642024-12-20T21:46:27.487ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = GET path = /v1/vpcs/{vpc} versions = all
28652024-12-20T21:46:27.488ZDEBGtest_mgs_metrics (dropshot_external): registered endpoint local_addr = 127.0.0.1:33109 method = PUT path = /v1/vpcs/{vpc} versions = all
28662024-12-20T21:46:27.488ZINFOtest_mgs_metrics (dropshot_external): listening local_addr = 127.0.0.1:33109
28672024-12-20T21:46:27.488ZDEBGtest_mgs_metrics (dropshot_external): successfully registered DTrace USDT probes local_addr = 127.0.0.1:33109
28682024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = / versions = all
28692024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /access versions = all
28702024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /assets/{path} versions = all
28712024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /device/auth versions = all
28722024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /device/confirm versions = all
28732024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /device/success versions = all
28742024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /device/token versions = all
28752024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /device/verify versions = all
28762024-12-20T21:46:27.520ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /experimental/v1/probes versions = all
28772024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /experimental/v1/probes versions = all
28782024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /experimental/v1/probes/{probe} versions = all
28792024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /experimental/v1/probes/{probe} versions = all
28802024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /images versions = all
28812024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /login versions = all
28822024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /login/{silo_name}/local versions = all
28832024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
28842024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
28852024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
28862024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /lookup/{path} versions = all
28872024-12-20T21:46:27.521ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /projects/{path} versions = all
28882024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /projects-new versions = all
28892024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /settings/{path} versions = all
28902024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /system/{path} versions = all
28912024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /utilization versions = all
28922024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/certificates versions = all
28932024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/certificates versions = all
28942024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/certificates/{certificate} versions = all
28952024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/certificates/{certificate} versions = all
28962024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/disks versions = all
28972024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/disks versions = all
28982024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/disks/{disk} versions = all
28992024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/disks/{disk} versions = all
29002024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/disks/{disk}/bulk-write versions = all
29012024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
29022024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
29032024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/disks/{disk}/finalize versions = all
29042024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
29052024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/floating-ips versions = all
29062024-12-20T21:46:27.522ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/floating-ips versions = all
29072024-12-20T21:46:27.523ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
29082024-12-20T21:46:27.523ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/floating-ips/{floating_ip} versions = all
29092024-12-20T21:46:27.523ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
29102024-12-20T21:46:27.523ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
29112024-12-20T21:46:27.523ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
29122024-12-20T21:46:27.523ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/groups versions = all
29132024-12-20T21:46:27.527ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/groups/{group_id} versions = all
29142024-12-20T21:46:27.527ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/images versions = all
29152024-12-20T21:46:27.527ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/images versions = all
29162024-12-20T21:46:27.527ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/images/{image} versions = all
29172024-12-20T21:46:27.527ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/images/{image} versions = all
29182024-12-20T21:46:27.527ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/images/{image}/demote versions = all
29192024-12-20T21:46:27.527ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/images/{image}/promote versions = all
29202024-12-20T21:46:27.527ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/instances versions = all
29212024-12-20T21:46:27.528ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/instances versions = all
29222024-12-20T21:46:27.528ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/instances/{instance} versions = all
29232024-12-20T21:46:27.528ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/instances/{instance} versions = all
29242024-12-20T21:46:27.528ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/instances/{instance} versions = all
29252024-12-20T21:46:27.528ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/instances/{instance}/disks versions = all
29262024-12-20T21:46:27.528ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/instances/{instance}/disks/attach versions = all
29272024-12-20T21:46:27.528ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/instances/{instance}/disks/detach versions = all
29282024-12-20T21:46:27.529ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/instances/{instance}/external-ips versions = all
29292024-12-20T21:46:27.529ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
29302024-12-20T21:46:27.529ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
29312024-12-20T21:46:27.529ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/instances/{instance}/reboot versions = all
29322024-12-20T21:46:27.529ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/instances/{instance}/serial-console versions = all
29332024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
29342024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
29352024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/instances/{instance}/start versions = all
29362024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/instances/{instance}/stop versions = all
29372024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/internet-gateway-ip-addresses versions = all
29382024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/internet-gateway-ip-addresses versions = all
29392024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
29402024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/internet-gateway-ip-pools versions = all
29412024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/internet-gateway-ip-pools versions = all
29422024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
29432024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/internet-gateways versions = all
29442024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/internet-gateways versions = all
29452024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
29462024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/internet-gateways/{gateway} versions = all
29472024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/ip-pools versions = all
29482024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/ip-pools/{pool} versions = all
29492024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/login/{silo_name}/local versions = all
29502024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/logout versions = all
29512024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/me versions = all
29522024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/me/groups versions = all
29532024-12-20T21:46:27.530ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/me/ssh-keys versions = all
29542024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/me/ssh-keys versions = all
29552024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
29562024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
29572024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/metrics/{metric_name} versions = all
29582024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/network-interfaces versions = all
29592024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/network-interfaces versions = all
29602024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/network-interfaces/{interface} versions = all
29612024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/network-interfaces/{interface} versions = all
29622024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/network-interfaces/{interface} versions = all
29632024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/ping versions = all
29642024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/policy versions = all
29652024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/policy versions = all
29662024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/projects versions = all
29672024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/projects versions = all
29682024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/projects/{project} versions = all
29692024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/projects/{project} versions = all
29702024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/projects/{project} versions = all
29712024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/projects/{project}/policy versions = all
29722024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/projects/{project}/policy versions = all
29732024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/snapshots versions = all
29742024-12-20T21:46:27.531ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/snapshots versions = all
29752024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/snapshots/{snapshot} versions = all
29762024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/snapshots/{snapshot} versions = all
29772024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/disks versions = all
29782024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
29792024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/racks versions = all
29802024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
29812024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/sleds versions = all
29822024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/hardware/sleds versions = all
29832024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
29842024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
29852024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
29862024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
29872024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
29882024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/switch-port versions = all
29892024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
29902024-12-20T21:46:27.533ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
29912024-12-20T21:46:27.534ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
29922024-12-20T21:46:27.534ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/switches versions = all
29932024-12-20T21:46:27.534ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
29942024-12-20T21:46:27.534ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/identity-providers versions = all
29952024-12-20T21:46:27.535ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/identity-providers/local/users versions = all
29962024-12-20T21:46:27.535ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
29972024-12-20T21:46:27.535ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
29982024-12-20T21:46:27.535ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/identity-providers/saml versions = all
29992024-12-20T21:46:27.535ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
30002024-12-20T21:46:27.535ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/ip-pools versions = all
30012024-12-20T21:46:27.536ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/ip-pools versions = all
30022024-12-20T21:46:27.536ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
30032024-12-20T21:46:27.536ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/ip-pools/{pool} versions = all
30042024-12-20T21:46:27.536ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/ip-pools/{pool} versions = all
30052024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
30062024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
30072024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
30082024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
30092024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
30102024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
30112024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
30122024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
30132024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/ip-pools-service versions = all
30142024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/ip-pools-service/ranges versions = all
30152024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
30162024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
30172024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/metrics/{metric_name} versions = all
30182024-12-20T21:46:27.537ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/address-lot versions = all
30192024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/networking/address-lot versions = all
30202024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
30212024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
30222024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/allow-list versions = all
30232024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/networking/allow-list versions = all
30242024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/networking/bfd-disable versions = all
30252024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/networking/bfd-enable versions = all
30262024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/bfd-status versions = all
30272024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/networking/bgp versions = all
30282024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/bgp versions = all
30292024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/networking/bgp versions = all
30302024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/bgp-announce-set versions = all
30312024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
30322024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
30332024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
30342024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/bgp-exported versions = all
30352024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/bgp-message-history versions = all
30362024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
30372024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/bgp-status versions = all
30382024-12-20T21:46:27.538ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/loopback-address versions = all
30392024-12-20T21:46:27.539ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/networking/loopback-address versions = all
30402024-12-20T21:46:27.539ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
30412024-12-20T21:46:27.539ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
30422024-12-20T21:46:27.539ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/switch-port-settings versions = all
30432024-12-20T21:46:27.539ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/networking/switch-port-settings versions = all
30442024-12-20T21:46:27.539ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
30452024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/policy versions = all
30462024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/policy versions = all
30472024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/roles versions = all
30482024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/roles/{role_name} versions = all
30492024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/silo-quotas versions = all
30502024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/silos versions = all
30512024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/silos versions = all
30522024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/system/silos/{silo} versions = all
30532024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/silos/{silo} versions = all
30542024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
30552024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/silos/{silo}/policy versions = all
30562024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/silos/{silo}/policy versions = all
30572024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/silos/{silo}/quotas versions = all
30582024-12-20T21:46:27.540ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
30592024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/system/timeseries/query versions = all
30602024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/timeseries/schemas versions = all
30612024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/system/update/repository versions = all
30622024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/update/repository/{system_version} versions = all
30632024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/users versions = all
30642024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/users/{user_id} versions = all
30652024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/users-builtin versions = all
30662024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/users-builtin/{user} versions = all
30672024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/utilization/silos versions = all
30682024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/system/utilization/silos/{silo} versions = all
30692024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/timeseries/query versions = all
30702024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/users versions = all
30712024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/utilization versions = all
30722024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpc-firewall-rules versions = all
30732024-12-20T21:46:27.541ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/vpc-firewall-rules versions = all
30742024-12-20T21:46:27.542ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpc-router-routes versions = all
30752024-12-20T21:46:27.542ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/vpc-router-routes versions = all
30762024-12-20T21:46:27.542ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
30772024-12-20T21:46:27.542ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpc-router-routes/{route} versions = all
30782024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/vpc-router-routes/{route} versions = all
30792024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpc-routers versions = all
30802024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/vpc-routers versions = all
30812024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/vpc-routers/{router} versions = all
30822024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpc-routers/{router} versions = all
30832024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/vpc-routers/{router} versions = all
30842024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpc-subnets versions = all
30852024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/vpc-subnets versions = all
30862024-12-20T21:46:27.543ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
30872024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpc-subnets/{subnet} versions = all
30882024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
30892024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
30902024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpcs versions = all
30912024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = POST path = /v1/vpcs versions = all
30922024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = DELETE path = /v1/vpcs/{vpc} versions = all
30932024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = GET path = /v1/vpcs/{vpc} versions = all
30942024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:38671 method = PUT path = /v1/vpcs/{vpc} versions = all
30952024-12-20T21:46:27.544ZINFOtest_mgs_metrics (dropshot_external_techport): listening local_addr = 127.0.0.1:38671
30962024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = 127.0.0.1:38671
30972024-12-20T21:46:27.544ZDEBGtest_mgs_metrics (nexus-producer-server): registered DTrace probes
30982024-12-20T21:46:27.545ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = 127.0.0.1:46305 method = GET path = /{producer_id} versions = all
30992024-12-20T21:46:27.545ZINFOtest_mgs_metrics (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = 127.0.0.1:46305
31002024-12-20T21:46:27.545ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = 127.0.0.1:46305
31012024-12-20T21:46:27.545ZDEBGtest_mgs_metrics (nexus-producer-server): Requested any available port, Dropshot server has been bound to 127.0.0.1:46305
31022024-12-20T21:46:27.545ZDEBGtest_mgs_metrics (nexus-producer-server): Nexus IP provided explicitly, will use it addr = 127.0.0.1:38423
31032024-12-20T21:46:27.545ZDEBGtest_mgs_metrics (nexus-producer-server): starting producer registration task
31042024-12-20T21:46:27.545ZINFOtest_mgs_metrics (nexus-producer-server): starting oximeter metric producer server address = 127.0.0.1:46305 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = cd9a0a31-a0ff-4b4d-bec4-e25b5f1a3176
31052024-12-20T21:46:27.546ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31062024-12-20T21:46:27.546ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
31072024-12-20T21:46:27.546ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31082024-12-20T21:46:27.546ZDEBGtest_mgs_metrics (producer-registration-task): registering / renewing oximeter producer lease with Nexus
31092024-12-20T21:46:27.546ZDEBGtest_mgs_metrics (producer-registration-task): using nexus address for registration addr = 127.0.0.1:38423
31102024-12-20T21:46:27.579ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:38423/metrics/producers
31112024-12-20T21:46:27.580ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54976
31122024-12-20T21:46:27.581ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54976 req_id = 0b54ab59-d3ca-463b-bcc0-40f427a800df uri = /metrics/producers
31132024-12-20T21:46:27.582ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54976 req_id = 0b54ab59-d3ca-463b-bcc0-40f427a800df resource = Database uri = /metrics/producers
31142024-12-20T21:46:27.582ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54976 req_id = 0b54ab59-d3ca-463b-bcc0-40f427a800df roles = RoleSet { roles: {} } uri = /metrics/producers
31152024-12-20T21:46:27.583ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54976 req_id = 0b54ab59-d3ca-463b-bcc0-40f427a800df resource = Database result = Ok(()) uri = /metrics/producers
31162024-12-20T21:46:27.583ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
31172024-12-20T21:46:27.583ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31182024-12-20T21:46:27.583ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31192024-12-20T21:46:27.583ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
31202024-12-20T21:46:27.584ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31212024-12-20T21:46:27.590ZDEBGtest_mgs_metrics (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:45883 result = Ok(Response { url: "http://[::1]:45883/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5b13006-4fea-4df9-89ca-ab46fe7ba686", "content-length": "723", "date": "Fri, 20 Dec 2024 21:46:27 GMT"} })
31222024-12-20T21:46:27.591ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
31232024-12-20T21:46:27.591ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31242024-12-20T21:46:27.591ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
31252024-12-20T21:46:27.591ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
31262024-12-20T21:46:27.591ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
31272024-12-20T21:46:27.595ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31282024-12-20T21:46:27.600ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
31292024-12-20T21:46:27.600ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) }
31302024-12-20T21:46:27.601ZTRACtest_mgs_metrics (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31312024-12-20T21:46:27.601ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31322024-12-20T21:46:27.601ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
31332024-12-20T21:46:27.602ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31342024-12-20T21:46:27.602ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31352024-12-20T21:46:27.606ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
31362024-12-20T21:46:27.606ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
31372024-12-20T21:46:27.606ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31382024-12-20T21:46:27.606ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
31392024-12-20T21:46:27.606ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
31402024-12-20T21:46:27.607ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
31412024-12-20T21:46:27.609ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31422024-12-20T21:46:27.614ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: 039be560-54cc-49e3-88df-1a29dadbf913, lookup_type: ById(039be560-54cc-49e3-88df-1a29dadbf913) } result = Ok(())
31432024-12-20T21:46:27.615ZTRACtest_mgs_metrics (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31442024-12-20T21:46:27.615ZDEBGtest_mgs_metrics (InternalApi): roles roles = RoleSet { roles: {} }
31452024-12-20T21:46:27.615ZDEBGtest_mgs_metrics (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31462024-12-20T21:46:27.615ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31472024-12-20T21:46:27.632ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
31482024-12-20T21:46:27.632ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
31492024-12-20T21:46:27.632ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
31502024-12-20T21:46:27.633ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
31512024-12-20T21:46:27.776ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:34315
31522024-12-20T21:46:27.776ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = http://[::1]:34315 background_task = inventory_collection body = None method = GET uri = http://[::1]:34315/inventory
31532024-12-20T21:46:27.777ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 196168 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54976 req_id = 0b54ab59-d3ca-463b-bcc0-40f427a800df response_code = 503 uri = /metrics/producers
31542024-12-20T21:46:27.778ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34315 remote_addr = [::1]:57970 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913
31552024-12-20T21:46:27.778ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:38423/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "0b54ab59-d3ca-463b-bcc0-40f427a800df", "content-length": "133", "date": "Fri, 20 Dec 2024 21:46:27 GMT"} })
31562024-12-20T21:46:27.780ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 238.499937ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"0b54ab59-d3ca-463b-bcc0-40f427a800df\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 20 Dec 2024 21:46:27 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"0b54ab59-d3ca-463b-bcc0-40f427a800df\\" }" file = oximeter/producer/src/lib.rs:420
31572024-12-20T21:46:27.781ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34315 method = GET remote_addr = [::1]:57970 req_id = 1d2c9fa6-bc6e-45fe-9855-9ca3bedfc400 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
31582024-12-20T21:46:27.782ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 586 local_addr = [::1]:34315 method = GET remote_addr = [::1]:57970 req_id = 1d2c9fa6-bc6e-45fe-9855-9ca3bedfc400 response_code = 200 sled_id = 039be560-54cc-49e3-88df-1a29dadbf913 uri = /inventory
31592024-12-20T21:46:27.783ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = http://[::1]:34315 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:34315/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d2c9fa6-bc6e-45fe-9855-9ca3bedfc400", "content-length": "4237", "date": "Fri, 20 Dec 2024 21:46:27 GMT"} })
31602024-12-20T21:46:27.825ZDEBGtest_mgs_metrics (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:34047
31612024-12-20T21:46:27.825ZDEBGtest_mgs_metrics (BackgroundTasks): client request SledAgent = http://[::1]:34047 background_task = inventory_collection body = None method = GET uri = http://[::1]:34047/inventory
31622024-12-20T21:46:27.825ZINFOtest_mgs_metrics (dropshot): accepted connection local_addr = [::1]:34047 remote_addr = [::1]:41270 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
31632024-12-20T21:46:27.826ZDEBGtest_mgs_metrics (init_with_steps): Running step start_oximeter
31642024-12-20T21:46:27.826ZDEBGtest_mgs_metrics: Starting Oximeter
31652024-12-20T21:46:27.826ZDEBGtest_mgs_metrics (oximeter): registered DTrace probes
31662024-12-20T21:46:27.826ZINFOtest_mgs_metrics (oximeter): starting oximeter server file = oximeter/collector/src/lib.rs:236
31672024-12-20T21:46:27.827ZDEBGtest_mgs_metrics (oximeter): creating ClickHouse client
31682024-12-20T21:46:27.827ZDEBGtest_mgs_metrics (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
31692024-12-20T21:46:27.827ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = SELECT MAX(value) AS max_version FROM oximeter.version;
31702024-12-20T21:46:27.827ZTRACtest_mgs_metrics (dropshot): incoming request local_addr = [::1]:34047 method = GET remote_addr = [::1]:41270 req_id = 7c5a104e-3d65-4d99-98b9-57fe33ebacf7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
31712024-12-20T21:46:27.828ZINFOtest_mgs_metrics (dropshot): request completed latency_us = 1223 local_addr = [::1]:34047 method = GET remote_addr = [::1]:41270 req_id = 7c5a104e-3d65-4d99-98b9-57fe33ebacf7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
31722024-12-20T21:46:27.829ZDEBGtest_mgs_metrics (BackgroundTasks): client response SledAgent = http://[::1]:34047 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:34047/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c5a104e-3d65-4d99-98b9-57fe33ebacf7", "content-length": "6315", "date": "Fri, 20 Dec 2024 21:46:27 GMT"} })
31732024-12-20T21:46:27.830ZDEBGtest_mgs_metrics (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
31742024-12-20T21:46:27.831ZDEBGtest_mgs_metrics (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
31752024-12-20T21:46:27.831ZDEBGtest_mgs_metrics (BackgroundTasks): finished collection background_task = inventory_collection
31762024-12-20T21:46:27.832ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
31772024-12-20T21:46:27.833ZTRACtest_mgs_metrics (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31782024-12-20T21:46:27.833ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
31792024-12-20T21:46:27.833ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
31802024-12-20T21:46:27.834ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
31812024-12-20T21:46:27.840ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31822024-12-20T21:46:27.844ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
31832024-12-20T21:46:27.845ZTRACtest_mgs_metrics (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
31842024-12-20T21:46:27.845ZDEBGtest_mgs_metrics (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
31852024-12-20T21:46:27.845ZDEBGtest_mgs_metrics (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
31862024-12-20T21:46:27.850ZWARNtest_mgs_metrics (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:823 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
31872024-12-20T21:46:27.850ZDEBGtest_mgs_metrics (oximeter-agent): oximeter database does not exist, creating collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31882024-12-20T21:46:27.850ZINFOtest_mgs_metrics (clickhouse-client): reading db version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:732 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
31892024-12-20T21:46:27.851ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = SELECT MAX(value) AS max_version FROM oximeter.version;
31902024-12-20T21:46:27.861ZWARNtest_mgs_metrics (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:823 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
31912024-12-20T21:46:27.861ZINFOtest_mgs_metrics (clickhouse-client): read oximeter database version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:736 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 version = 0
31922024-12-20T21:46:27.861ZINFOtest_mgs_metrics (clickhouse-client): wiping and re-initializing oximeter schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:741 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
31932024-12-20T21:46:27.861ZDEBGtest_mgs_metrics (clickhouse-client): wiping ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
31942024-12-20T21:46:27.861ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = DROP DATABASE IF EXISTS oximeter
31952024-12-20T21:46:27.863ZDEBGtest_mgs_metrics (clickhouse-client): initializing ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
31962024-12-20T21:46:27.863ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = CREATE DATABASE IF NOT EXISTS oximeter
31972024-12-20T21:46:27.873ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\n/* The version table contains metadata about the `oximeter` database */\nCREATE TABLE IF NOT EXISTS oximeter.version\n(\n value UInt64,\n timestamp DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (value, timestamp)
31982024-12-20T21:46:27.889ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\n/* The measurement tables contain all individual samples from each timeseries.\n *\n * Each table stores a single datum type, and otherwise contains nearly the same\n * structure. The primary sorting key is on the timeseries name, key, and then\n * timestamp, so that all timeseries from the same schema are grouped, followed\n * by all samples from the same timeseries.\n *\n * This reflects that one usually looks up the _key_ in one or more field table,\n * and then uses that to index quickly into the measurements tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Bool)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
31992024-12-20T21:46:27.903ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32002024-12-20T21:46:27.915ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32012024-12-20T21:46:27.937ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32022024-12-20T21:46:27.953ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32032024-12-20T21:46:27.977ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32042024-12-20T21:46:28.005ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32052024-12-20T21:46:28.051ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:38423/metrics/producers
32062024-12-20T21:46:28.054ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32072024-12-20T21:46:28.054ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54978
32082024-12-20T21:46:28.057ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54978 req_id = 77f3eec6-f75b-47fd-8f23-eacf769c115c uri = /metrics/producers
32092024-12-20T21:46:28.057ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54978 req_id = 77f3eec6-f75b-47fd-8f23-eacf769c115c resource = Database uri = /metrics/producers
32102024-12-20T21:46:28.059ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54978 req_id = 77f3eec6-f75b-47fd-8f23-eacf769c115c roles = RoleSet { roles: {} } uri = /metrics/producers
32112024-12-20T21:46:28.059ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54978 req_id = 77f3eec6-f75b-47fd-8f23-eacf769c115c resource = Database result = Ok(()) uri = /metrics/producers
32122024-12-20T21:46:28.071ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32132024-12-20T21:46:28.083ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32142024-12-20T21:46:28.101ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32152024-12-20T21:46:28.116ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(String)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32162024-12-20T21:46:28.131ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bytes\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Right now we can't unambiguously record a nullable byte array.\n * Arrays cannot be nested in `Nullable()` types, and encoding the array as\n * a string isn't palatable for a few reasons.\n * See: https://github.com/oxidecomputer/omicron/issues/4551 for more\n * details.\n */\n datum Array(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32172024-12-20T21:46:28.132ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 75030 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54978 req_id = 77f3eec6-f75b-47fd-8f23-eacf769c115c response_code = 503 uri = /metrics/producers
32182024-12-20T21:46:28.134ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:38423/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "77f3eec6-f75b-47fd-8f23-eacf769c115c", "content-length": "133", "date": "Fri, 20 Dec 2024 21:46:27 GMT"} })
32192024-12-20T21:46:28.136ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 484.120127ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"77f3eec6-f75b-47fd-8f23-eacf769c115c\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 20 Dec 2024 21:46:27 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"77f3eec6-f75b-47fd-8f23-eacf769c115c\\" }" file = oximeter/producer/src/lib.rs:420
32202024-12-20T21:46:28.143ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativei64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32212024-12-20T21:46:28.155ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativeu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32222024-12-20T21:46:28.174ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32232024-12-20T21:46:28.193ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32242024-12-20T21:46:28.205ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Array types cannot be Nullable, see\n * https://clickhouse.com/docs/en/sql-reference/data-types/nullable\n * for more details.\n *\n * This means we need to use empty arrays to indicate a missing value. This\n * is unfortunate, and at this point relies on the fact that an\n * `oximeter::Histogram` cannot have zero bins. If that changes, we'll need\n * to figure out another way to represent missing samples here.\n */\n bins Array(Int8),\n counts Array(UInt64),\n min Int8,\n max Int8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32252024-12-20T21:46:28.220ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt8),\n counts Array(UInt64),\n min UInt8,\n max UInt8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32262024-12-20T21:46:28.231ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int16),\n counts Array(UInt64),\n min Int16,\n max Int16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32272024-12-20T21:46:28.242ZINFOtest_mgs_metrics (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 497bd94c-b421-460f-b19a-78515b850cda
32282024-12-20T21:46:28.242ZDEBGtest_mgs_metrics (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 497bd94c-b421-460f-b19a-78515b850cda time_started = 2024-12-20 21:46:26.747289 UTC
32292024-12-20T21:46:28.242ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.876197816s iteration = 4 status = {"collection_id":"497bd94c-b421-460f-b19a-78515b850cda","time_done":"2024-12-20 21:46:27.830805 UTC","time_started":"2024-12-20 21:46:26.747289 UTC"}
32302024-12-20T21:46:28.242ZDEBGtest_mgs_metrics (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Dependency
32312024-12-20T21:46:28.242ZDEBGtest_mgs_metrics (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 63.211\u{b5}s iteration = 3 status = {"error":"task disabled"}
32322024-12-20T21:46:28.248ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt16),\n counts Array(UInt64),\n min UInt16,\n max UInt16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32332024-12-20T21:46:28.264ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int32),\n counts Array(UInt64),\n min Int32,\n max Int32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32342024-12-20T21:46:28.290ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt32),\n counts Array(UInt64),\n min UInt32,\n max UInt32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32352024-12-20T21:46:28.307ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int64),\n counts Array(UInt64),\n min Int64,\n max Int64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32362024-12-20T21:46:28.319ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt64),\n counts Array(UInt64),\n min UInt64,\n max UInt64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32372024-12-20T21:46:28.331ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float32),\n counts Array(UInt64),\n min Float32,\n max Float32,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32382024-12-20T21:46:28.344ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float64),\n counts Array(UInt64),\n min Float64,\n max Float64,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
32392024-12-20T21:46:28.357ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\n/* The field tables store named dimensions of each timeseries.\n *\n * As with the measurement tables, there is one field table for each field data\n * type. Fields are deduplicated by using the "replacing merge tree", though\n * this behavior **must not** be relied upon for query correctness.\n *\n * The index for the fields differs from the measurements, however. Rows are\n * sorted by timeseries name, then field name, field value, and finally\n * timeseries key. This reflects the most common pattern for looking them up:\n * by field name and possibly value, within a timeseries. The resulting keys are\n * usually then used to look up measurements.\n *\n * NOTE: We may want to consider a secondary index on these tables, sorting by\n * timeseries name and then key, since it would improve lookups where one\n * already has the key. Realistically though, these tables are quite small and\n * so performance benefits will be low in absolute terms.\n *\n * TTL: We use a materialized column to expire old field table records. This\n * column is generated automatically by the database whenever a new row is\n * inserted. It cannot be inserted directly, nor is it returned in a `SELECT *`\n * query. Since these tables are `ReplacingMergeTree`s, that means the last\n * record will remain during a deduplication, which will have the last\n * timestamp. ClickHouse will then expire old data for us, similar to the\n * measurement tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.fields_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Bool,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32402024-12-20T21:46:28.377ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32412024-12-20T21:46:28.394ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32422024-12-20T21:46:28.410ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32432024-12-20T21:46:28.425ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32442024-12-20T21:46:28.438ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32452024-12-20T21:46:28.450ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32462024-12-20T21:46:28.471ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32472024-12-20T21:46:28.484ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32482024-12-20T21:46:28.496ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_ipaddr\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value IPv6,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32492024-12-20T21:46:28.516ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value String,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32502024-12-20T21:46:28.530ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_uuid\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UUID,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
32512024-12-20T21:46:28.548ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = \n\n/* The timeseries schema table stores the extracted schema for the samples\n * oximeter collects.\n */\nCREATE TABLE IF NOT EXISTS oximeter.timeseries_schema\n(\n timeseries_name String,\n fields Nested(\n name String,\n type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'IpAddr' = 3,\n 'String' = 4,\n 'Uuid' = 6,\n 'I8' = 7,\n 'U8' = 8,\n 'I16' = 9,\n 'U16' = 10,\n 'I32' = 11,\n 'U32' = 12,\n 'U64' = 13\n ),\n source Enum(\n 'Target' = 1,\n 'Metric' = 2\n )\n ),\n datum_type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'F64' = 3,\n 'String' = 4,\n 'Bytes' = 5,\n 'CumulativeI64' = 6,\n 'CumulativeF64' = 7,\n 'HistogramI64' = 8,\n 'HistogramF64' = 9,\n 'I8' = 10,\n 'U8' = 11,\n 'I16' = 12,\n 'U16' = 13,\n 'I32' = 14,\n 'U32' = 15,\n 'U64' = 16,\n 'F32' = 17,\n 'CumulativeU64' = 18,\n 'CumulativeF32' = 19,\n 'HistogramI8' = 20,\n 'HistogramU8' = 21,\n 'HistogramI16' = 22,\n 'HistogramU16' = 23,\n 'HistogramI32' = 24,\n 'HistogramU32' = 25,\n 'HistogramU64' = 26,\n 'HistogramF32' = 27\n ),\n created DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, fields.name)
32522024-12-20T21:46:28.563ZINFOtest_mgs_metrics (clickhouse-client): inserting current version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:764 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 version = 13
32532024-12-20T21:46:28.563ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = INSERT INTO oximeter.version (*) VALUES (13, now());
32542024-12-20T21:46:28.565ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39775 method = GET path = /info versions = all
32552024-12-20T21:46:28.566ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39775 method = GET path = /producers versions = all
32562024-12-20T21:46:28.566ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39775 method = DELETE path = /producers/{producer_id} versions = all
32572024-12-20T21:46:28.566ZDEBGtest_mgs_metrics (dropshot): registered endpoint local_addr = [::1]:39775 method = GET path = /producers/{producer_id} versions = all
32582024-12-20T21:46:28.566ZINFOtest_mgs_metrics (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [::1]:39775
32592024-12-20T21:46:28.566ZDEBGtest_mgs_metrics (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:39775
32602024-12-20T21:46:28.566ZDEBGtest_mgs_metrics (oximeter): registered USDT probes
32612024-12-20T21:46:28.566ZDEBGtest_mgs_metrics (oximeter): contacting nexus
32622024-12-20T21:46:29.095ZDEBGtest_mgs_metrics (oximeter): client request body = Some(Body) method = POST uri = http://127.0.0.1:38423/metrics/collectors
32632024-12-20T21:46:29.202ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:38423/metrics/producers
32642024-12-20T21:46:29.204ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54982
32652024-12-20T21:46:29.204ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54984
32662024-12-20T21:46:29.205ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54984 req_id = e030adbc-7c2c-4e5b-a667-69f9548379e1 uri = /metrics/producers
32672024-12-20T21:46:29.207ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54984 req_id = e030adbc-7c2c-4e5b-a667-69f9548379e1 resource = Database uri = /metrics/producers
32682024-12-20T21:46:29.207ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54984 req_id = e030adbc-7c2c-4e5b-a667-69f9548379e1 roles = RoleSet { roles: {} } uri = /metrics/producers
32692024-12-20T21:46:29.208ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54984 req_id = e030adbc-7c2c-4e5b-a667-69f9548379e1 resource = Database result = Ok(()) uri = /metrics/producers
32702024-12-20T21:46:29.211ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 90 19 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:63246 req_id = e586620c-451a-43fe-9802-36f8fd31dff9
32712024-12-20T21:46:29.211ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36889,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63246 req_id = e586620c-451a-43fe-9802-36f8fd31dff9
32722024-12-20T21:46:29.214ZDEBGtest_mgs_metrics (store): query key key = _crucible-pantry._tcp
32732024-12-20T21:46:29.214ZDEBGtest_mgs_metrics (store): query key key = bca45c56-47e5-420e-82e5-fce0647d0ba3.host
32742024-12-20T21:46:29.214ZDEBGtest_mgs_metrics (dns): dns response additional_records = [Record { name_labels: Name("bca45c56-47e5-420e-82e5-fce0647d0ba3.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63246 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39291, target: Name("bca45c56-47e5-420e-82e5-fce0647d0ba3.host.control-plane.oxide.internal") })) }] req_id = e586620c-451a-43fe-9802-36f8fd31dff9
32752024-12-20T21:46:29.214ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54982 req_id = 753063ef-2a2f-49e7-b2bc-992668c18d35 uri = /metrics/collectors
32762024-12-20T21:46:29.214ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54982 req_id = 753063ef-2a2f-49e7-b2bc-992668c18d35 resource = Database uri = /metrics/collectors
32772024-12-20T21:46:29.214ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54982 req_id = 753063ef-2a2f-49e7-b2bc-992668c18d35 roles = RoleSet { roles: {} } uri = /metrics/collectors
32782024-12-20T21:46:29.214ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54982 req_id = 753063ef-2a2f-49e7-b2bc-992668c18d35 resource = Database result = Ok(()) uri = /metrics/collectors
32792024-12-20T21:46:29.229ZINFOtest_mgs_metrics (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 23858 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54984 req_id = e030adbc-7c2c-4e5b-a667-69f9548379e1 response_code = 503 uri = /metrics/producers
32802024-12-20T21:46:29.244ZTRACtest_mgs_metrics (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 02 0a 01 00 00 01 00 00 00 00 00 01 24 62 63 61 ............$bca\n0010: 34 35 63 35 36 2d 34 37 65 35 2d 34 32 30 65 2d 45c56-47e5-420e-\n0020: 38 32 65 35 2d 66 63 65 30 36 34 37 64 30 62 61 82e5-fce0647d0ba\n0030: 33 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 3.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:52270 req_id = 6042028b-4809-40ac-a754-974ed4e755ad
32812024-12-20T21:46:29.244ZDEBGtest_mgs_metrics (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 522,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("bca45c56-47e5-420e-82e5-fce0647d0ba3.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("bca45c56-47e5-420e-82e5-fce0647d0ba3.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 98,\n 99,\n 97,\n 52,\n 53,\n 99,\n 53,\n 54,\n 45,\n 52,\n 55,\n 101,\n 53,\n 45,\n 52,\n 50,\n 48,\n 101,\n 45,\n 56,\n 50,\n 101,\n 53,\n 45,\n 102,\n 99,\n 101,\n 48,\n 54,\n 52,\n 55,\n 100,\n 48,\n 98,\n 97,\n 51,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52270 req_id = 6042028b-4809-40ac-a754-974ed4e755ad
32822024-12-20T21:46:29.244ZDEBGtest_mgs_metrics (store): query key key = bca45c56-47e5-420e-82e5-fce0647d0ba3.host
32832024-12-20T21:46:29.244ZDEBGtest_mgs_metrics (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52270 query = LowerQuery { name: LowerName(Name("bca45c56-47e5-420e-82e5-fce0647d0ba3.host.control-plane.oxide.internal.")), original: Query { name: Name("bca45c56-47e5-420e-82e5-fce0647d0ba3.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("bca45c56-47e5-420e-82e5-fce0647d0ba3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6042028b-4809-40ac-a754-974ed4e755ad
32842024-12-20T21:46:29.247ZDEBGtest_mgs_metrics (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:38423/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "e030adbc-7c2c-4e5b-a667-69f9548379e1", "content-length": "133", "date": "Fri, 20 Dec 2024 21:46:29 GMT"} })
32852024-12-20T21:46:29.249ZWARNtest_mgs_metrics (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.268042299s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"e030adbc-7c2c-4e5b-a667-69f9548379e1\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 20 Dec 2024 21:46:29 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"e030adbc-7c2c-4e5b-a667-69f9548379e1\\" }" file = oximeter/producer/src/lib.rs:420
32862024-12-20T21:46:29.251ZINFOtest_mgs_metrics (nexus): registered new oximeter metric collection server address = [::1]:39775 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
32872024-12-20T21:46:29.252ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 41531 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54982 req_id = 753063ef-2a2f-49e7-b2bc-992668c18d35 response_code = 204 uri = /metrics/collectors
32882024-12-20T21:46:29.374ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:38423/metrics/collectors", status: 204, headers: {"x-request-id": "753063ef-2a2f-49e7-b2bc-992668c18d35", "date": "Fri, 20 Dec 2024 21:46:29 GMT"} })
32892024-12-20T21:46:29.376ZINFOtest_mgs_metrics (oximeter): oximeter registered with nexus file = oximeter/collector/src/lib.rs:383 id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
32902024-12-20T21:46:29.377ZDEBGtest_mgs_metrics (init_with_steps): Running step start_producer_server
32912024-12-20T21:46:29.377ZDEBGtest_mgs_metrics: Starting test metric Producer Server
32922024-12-20T21:46:29.446ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39291 remote_addr = [::1]:43532 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
32932024-12-20T21:46:29.447ZINFOtest_mgs_metrics (oximeter-agent): starting refresh list task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:471
32942024-12-20T21:46:29.633ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39291 method = GET remote_addr = [::1]:43532 req_id = d0d524eb-050f-49cf-99de-0f6f397070f6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
32952024-12-20T21:46:29.633ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54986
32962024-12-20T21:46:29.633ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54990
32972024-12-20T21:46:29.633ZINFOtest_mgs_metrics (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:474
32982024-12-20T21:46:29.633ZTRACtest_mgs_metrics (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32992024-12-20T21:46:29.644ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 10922 local_addr = [::1]:39291 method = GET remote_addr = [::1]:43532 req_id = d0d524eb-050f-49cf-99de-0f6f397070f6 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33002024-12-20T21:46:29.644ZDEBGtest_mgs_metrics (oximeter): client request body = None method = GET uri = http://127.0.0.1:38423/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
33012024-12-20T21:46:29.646ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54986 req_id = 11ea7e11-687d-4499-b108-c461f6214d47 uri = /metrics/producers
33022024-12-20T21:46:29.646ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54990 req_id = 2eb23c86-bd96-4f5c-b67c-ec577772f513 uri = /metrics/producers
33032024-12-20T21:46:29.647ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 3ba32890-b1ba-445b-8a06-03efb6c8210e uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
33042024-12-20T21:46:29.647ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54986 req_id = 11ea7e11-687d-4499-b108-c461f6214d47 resource = Database uri = /metrics/producers
33052024-12-20T21:46:29.647ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54986 req_id = 11ea7e11-687d-4499-b108-c461f6214d47 roles = RoleSet { roles: {} } uri = /metrics/producers
33062024-12-20T21:46:29.648ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54986 req_id = 11ea7e11-687d-4499-b108-c461f6214d47 resource = Database result = Ok(()) uri = /metrics/producers
33072024-12-20T21:46:29.649ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54990 req_id = 2eb23c86-bd96-4f5c-b67c-ec577772f513 resource = Database uri = /metrics/producers
33082024-12-20T21:46:29.649ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54990 req_id = 2eb23c86-bd96-4f5c-b67c-ec577772f513 roles = RoleSet { roles: {} } uri = /metrics/producers
33092024-12-20T21:46:29.650ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54990 req_id = 2eb23c86-bd96-4f5c-b67c-ec577772f513 resource = Database result = Ok(()) uri = /metrics/producers
33102024-12-20T21:46:29.650ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 3ba32890-b1ba-445b-8a06-03efb6c8210e resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
33112024-12-20T21:46:29.650ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 3ba32890-b1ba-445b-8a06-03efb6c8210e roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
33122024-12-20T21:46:29.653ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 3ba32890-b1ba-445b-8a06-03efb6c8210e resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
33132024-12-20T21:46:30.120ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39291 remote_addr = [::1]:43534 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33142024-12-20T21:46:30.120ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39291 remote_addr = [::1]:43536 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33152024-12-20T21:46:30.120ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39291 remote_addr = [::1]:43538 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33162024-12-20T21:46:30.121ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39291 remote_addr = [::1]:43540 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33172024-12-20T21:46:30.121ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39291 remote_addr = [::1]:43544 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33182024-12-20T21:46:30.121ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39291 remote_addr = [::1]:43546 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33192024-12-20T21:46:30.121ZINFOtest_mgs_metrics (dropshot): accepted connection kind = pantry local_addr = [::1]:39291 remote_addr = [::1]:43548 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
33202024-12-20T21:46:30.126ZINFOtest_mgs_metrics (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33212024-12-20T21:46:30.127ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 481303 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54986 req_id = 11ea7e11-687d-4499-b108-c461f6214d47 response_code = 201 uri = /metrics/producers
33222024-12-20T21:46:30.128ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39291 method = GET remote_addr = [::1]:43534 req_id = 06acd081-15c0-4765-8897-17cd28f6aaee sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33232024-12-20T21:46:30.128ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39291 method = GET remote_addr = [::1]:43536 req_id = c93d2ee6-99c6-4a35-957a-86f9e6ef9111 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33242024-12-20T21:46:30.128ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39291 method = GET remote_addr = [::1]:43538 req_id = 63ca466f-3265-4e6a-999e-f2db3b89c5f0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33252024-12-20T21:46:30.129ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39291 method = GET remote_addr = [::1]:43540 req_id = df9ed38f-75b4-40a4-8b68-5d65305b4b18 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33262024-12-20T21:46:30.129ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39291 method = GET remote_addr = [::1]:43544 req_id = b014657f-2252-42cb-a144-88ca305e1801 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33272024-12-20T21:46:30.129ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39291 method = GET remote_addr = [::1]:43546 req_id = ec8a299c-5b52-41d0-91c8-6586b59980ca sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33282024-12-20T21:46:30.129ZTRACtest_mgs_metrics (dropshot): incoming request kind = pantry local_addr = [::1]:39291 method = GET remote_addr = [::1]:43548 req_id = 5941452b-5d24-464e-b613-139b52a24bc3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33292024-12-20T21:46:30.134ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 6180 local_addr = [::1]:39291 method = GET remote_addr = [::1]:43534 req_id = 06acd081-15c0-4765-8897-17cd28f6aaee response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33302024-12-20T21:46:30.135ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 6125 local_addr = [::1]:39291 method = GET remote_addr = [::1]:43536 req_id = c93d2ee6-99c6-4a35-957a-86f9e6ef9111 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33312024-12-20T21:46:30.135ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 6054 local_addr = [::1]:39291 method = GET remote_addr = [::1]:43538 req_id = 63ca466f-3265-4e6a-999e-f2db3b89c5f0 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33322024-12-20T21:46:30.135ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 5994 local_addr = [::1]:39291 method = GET remote_addr = [::1]:43540 req_id = df9ed38f-75b4-40a4-8b68-5d65305b4b18 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33332024-12-20T21:46:30.135ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 5985 local_addr = [::1]:39291 method = GET remote_addr = [::1]:43544 req_id = b014657f-2252-42cb-a144-88ca305e1801 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33342024-12-20T21:46:30.135ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 5920 local_addr = [::1]:39291 method = GET remote_addr = [::1]:43546 req_id = ec8a299c-5b52-41d0-91c8-6586b59980ca response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33352024-12-20T21:46:30.136ZINFOtest_mgs_metrics (dropshot): request completed kind = pantry latency_us = 5991 local_addr = [::1]:39291 method = GET remote_addr = [::1]:43548 req_id = 5941452b-5d24-464e-b613-139b52a24bc3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
33362024-12-20T21:46:30.138ZINFOtest_mgs_metrics (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33372024-12-20T21:46:30.138ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 492046 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54990 req_id = 2eb23c86-bd96-4f5c-b67c-ec577772f513 response_code = 201 uri = /metrics/producers
33382024-12-20T21:46:30.154ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 507006 local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 3ba32890-b1ba-445b-8a06-03efb6c8210e response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
33392024-12-20T21:46:30.156ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:38423/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ba32890-b1ba-445b-8a06-03efb6c8210e", "content-length": "412", "date": "Fri, 20 Dec 2024 21:46:29 GMT"} })
33402024-12-20T21:46:30.156ZDEBGtest_mgs_metrics (oximeter): client request body = None method = GET uri = http://127.0.0.1:38423/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjZTM4MzM0ZS1hOGE2LTQ0ZTQtOTlmNy02YjlmMzFkOGIxOGEifX0%3D
33412024-12-20T21:46:30.156ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 76d613a8-e3b4-404a-9308-9015f4f9759e uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjZTM4MzM0ZS1hOGE2LTQ0ZTQtOTlmNy02YjlmMzFkOGIxOGEifX0%3D
33422024-12-20T21:46:30.163ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 76d613a8-e3b4-404a-9308-9015f4f9759e resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjZTM4MzM0ZS1hOGE2LTQ0ZTQtOTlmNy02YjlmMzFkOGIxOGEifX0%3D
33432024-12-20T21:46:30.164ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 76d613a8-e3b4-404a-9308-9015f4f9759e roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjZTM4MzM0ZS1hOGE2LTQ0ZTQtOTlmNy02YjlmMzFkOGIxOGEifX0%3D
33442024-12-20T21:46:30.164ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 76d613a8-e3b4-404a-9308-9015f4f9759e resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjZTM4MzM0ZS1hOGE2LTQ0ZTQtOTlmNy02YjlmMzFkOGIxOGEifX0%3D
33452024-12-20T21:46:30.184ZINFOtest_mgs_metrics (dropshot_internal): request completed latency_us = 27191 local_addr = 127.0.0.1:38423 method = GET remote_addr = 127.0.0.1:54982 req_id = 76d613a8-e3b4-404a-9308-9015f4f9759e response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjZTM4MzM0ZS1hOGE2LTQ0ZTQtOTlmNy02YjlmMzFkOGIxOGEifX0%3D
33462024-12-20T21:46:30.184ZDEBGtest_mgs_metrics (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:38423/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjZTM4MzM0ZS1hOGE2LTQ0ZTQtOTlmNy02YjlmMzFkOGIxOGEifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "76d613a8-e3b4-404a-9308-9015f4f9759e", "content-length": "29", "date": "Fri, 20 Dec 2024 21:46:29 GMT"} })
33472024-12-20T21:46:30.184ZDEBGtest_mgs_metrics (oximeter-agent): registered new metric producer address = [::1]:42569 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33482024-12-20T21:46:30.224ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33492024-12-20T21:46:30.229ZDEBGtest_mgs_metrics (oximeter-agent): registered new metric producer address = [::1]:45367 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33502024-12-20T21:46:30.272ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33512024-12-20T21:46:30.297ZINFOtest_mgs_metrics (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:549 n_current_tasks = 2 n_pruned_tasks = 0
33522024-12-20T21:46:30.298ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33532024-12-20T21:46:30.298ZTRACtest_mgs_metrics (collection-task): forwarded explicit request to collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33542024-12-20T21:46:30.298ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33552024-12-20T21:46:30.298ZTRACtest_mgs_metrics (collection-task): forwarded explicit request to collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33562024-12-20T21:46:30.303ZDEBGtest_mgs_metrics (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33572024-12-20T21:46:30.304ZDEBGtest_mgs_metrics (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33582024-12-20T21:46:30.319ZTRACtest_mgs_metrics (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33592024-12-20T21:46:30.320ZDEBGtest_mgs_metrics (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33602024-12-20T21:46:30.321ZTRACtest_mgs_metrics (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33612024-12-20T21:46:30.321ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33622024-12-20T21:46:30.321ZTRACtest_mgs_metrics (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33632024-12-20T21:46:30.321ZDEBGtest_mgs_metrics (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 2 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33642024-12-20T21:46:30.321ZTRACtest_mgs_metrics (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33652024-12-20T21:46:30.321ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33662024-12-20T21:46:30.321ZDEBGtest_mgs_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
33672024-12-20T21:46:30.321ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
33682024-12-20T21:46:30.321ZDEBGtest_mgs_metrics (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
33692024-12-20T21:46:30.321ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
33702024-12-20T21:46:30.344ZTRACtest_mgs_metrics (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
33712024-12-20T21:46:30.344ZDEBGtest_mgs_metrics (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 schema = TimeseriesSchema { timeseries_name: TimeseriesName("integration_target:integration_metric"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "metric_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "target_name", field_type: String, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2024-12-20T21:46:30.321638360Z } timeseries_name = integration_target:integration_metric
33722024-12-20T21:46:30.344ZDEBGtest_mgs_metrics (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
33732024-12-20T21:46:30.344ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_columns = 6 n_rows = 1 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
33742024-12-20T21:46:30.352ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
33752024-12-20T21:46:30.362ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_rows = 2 table_name = fields_string
33762024-12-20T21:46:30.362ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
33772024-12-20T21:46:30.371ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_rows = 1 table_name = measurements_i64
33782024-12-20T21:46:30.371ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
33792024-12-20T21:46:30.371ZDEBGtest_mgs_metrics (results-sink): inserting 0 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
33802024-12-20T21:46:30.371ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 0 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
33812024-12-20T21:46:30.371ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
33822024-12-20T21:46:30.371ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33832024-12-20T21:46:30.372ZTRACtest_mgs_metrics (collection-task): forwarded explicit request to collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33842024-12-20T21:46:30.372ZDEBGtest_mgs_metrics (collection-task): collection task received explicit request to collect collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33852024-12-20T21:46:30.372ZTRACtest_mgs_metrics (collection-task): forwarded explicit request to collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33862024-12-20T21:46:30.373ZDEBGtest_mgs_metrics (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33872024-12-20T21:46:30.373ZDEBGtest_mgs_metrics (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33882024-12-20T21:46:30.373ZTRACtest_mgs_metrics (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33892024-12-20T21:46:30.375ZDEBGtest_mgs_metrics (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33902024-12-20T21:46:30.375ZTRACtest_mgs_metrics (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33912024-12-20T21:46:30.375ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33922024-12-20T21:46:30.375ZTRACtest_mgs_metrics (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33932024-12-20T21:46:30.375ZDEBGtest_mgs_metrics (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 2 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33942024-12-20T21:46:30.375ZTRACtest_mgs_metrics (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33952024-12-20T21:46:30.375ZTRACtest_mgs_metrics (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ce38334e-a8a6-44e4-99f7-6b9f31d8b18a
33962024-12-20T21:46:30.376ZDEBGtest_mgs_metrics (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
33972024-12-20T21:46:30.376ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
33982024-12-20T21:46:30.376ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
33992024-12-20T21:46:30.380ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_rows = 2 table_name = fields_string
34002024-12-20T21:46:30.380ZTRACtest_mgs_metrics (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
34012024-12-20T21:46:30.385ZDEBGtest_mgs_metrics (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3 n_rows = 1 table_name = measurements_i64
34022024-12-20T21:46:30.386ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
34032024-12-20T21:46:30.386ZDEBGtest_mgs_metrics (results-sink): inserting 0 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
34042024-12-20T21:46:30.386ZDEBGtest_mgs_metrics (clickhouse-client): unrolling 0 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 73d24f42-5ea4-4caa-aa3a-84958ffb16d3
34052024-12-20T21:46:30.386ZTRACtest_mgs_metrics (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
34062024-12-20T21:46:30.386ZINFOtest_mgs_metrics (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:33109/v1/system/timeseries/query
34072024-12-20T21:46:30.386ZINFOtest_mgs_metrics (dropshot_external): accepted connection local_addr = 127.0.0.1:33109 remote_addr = 127.0.0.1:55320
34082024-12-20T21:46:30.388ZTRACtest_mgs_metrics (dropshot_external): incoming request local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 uri = /v1/system/timeseries/query
34092024-12-20T21:46:30.388ZTRACtest_mgs_metrics (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 uri = /v1/system/timeseries/query
34102024-12-20T21:46:30.388ZDEBGtest_mgs_metrics (dropshot_external): looking up silo for user local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/timeseries/query
34112024-12-20T21:46:30.388ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34122024-12-20T21:46:30.388ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
34132024-12-20T21:46:30.388ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34142024-12-20T21:46:30.397ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34152024-12-20T21:46:30.398ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
34162024-12-20T21:46:30.399ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34172024-12-20T21:46:30.412ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
34182024-12-20T21:46:30.412ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
34192024-12-20T21:46:30.412ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34202024-12-20T21:46:30.412ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
34212024-12-20T21:46:30.412ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34222024-12-20T21:46:30.415ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34232024-12-20T21:46:30.415ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34242024-12-20T21:46:30.415ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
34252024-12-20T21:46:30.416ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34262024-12-20T21:46:30.418ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
34272024-12-20T21:46:30.422ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
34282024-12-20T21:46:30.422ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34292024-12-20T21:46:30.422ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
34302024-12-20T21:46:30.423ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34312024-12-20T21:46:30.429ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
34322024-12-20T21:46:30.429ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
34332024-12-20T21:46:30.429ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34342024-12-20T21:46:30.429ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
34352024-12-20T21:46:30.429ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34362024-12-20T21:46:30.451ZTRACtest_mgs_metrics (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34372024-12-20T21:46:30.451ZTRACtest_mgs_metrics (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34382024-12-20T21:46:30.451ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {} }
34392024-12-20T21:46:30.452ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34402024-12-20T21:46:30.478ZDEBGtest_mgs_metrics (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
34412024-12-20T21:46:30.484ZDEBGtest_mgs_metrics (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
34422024-12-20T21:46:30.484ZTRACtest_mgs_metrics (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 uri = /v1/system/timeseries/query
34432024-12-20T21:46:30.484ZTRACtest_mgs_metrics (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 resource = Fleet uri = /v1/system/timeseries/query
34442024-12-20T21:46:30.484ZTRACtest_mgs_metrics (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/timeseries/query
34452024-12-20T21:46:30.485ZTRACtest_mgs_metrics (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 resource = Database uri = /v1/system/timeseries/query
34462024-12-20T21:46:30.485ZDEBGtest_mgs_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 roles = RoleSet { roles: {} } uri = /v1/system/timeseries/query
34472024-12-20T21:46:30.486ZDEBGtest_mgs_metrics (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 resource = Database result = Ok(()) uri = /v1/system/timeseries/query
34482024-12-20T21:46:30.503ZDEBGtest_mgs_metrics (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/timeseries/query
34492024-12-20T21:46:30.516ZDEBGtest_mgs_metrics (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 resource = Fleet result = Ok(()) uri = /v1/system/timeseries/query
34502024-12-20T21:46:30.517ZDEBGtest_mgs_metrics (clickhouse-client): parsed OxQL query id = e30132f4-6bad-4aa8-9298-df473cc1db82 parsed_query = Query { parsed: Query { ops: [Basic(Get(TimeseriesName("hardware_component:temperature")))] }, end_time: 2024-12-20T21:46:30.516771287Z } query = get hardware_component:temperature query_id = f142d1a5-822c-4e56-91fa-5343514ccdcb
34512024-12-20T21:46:30.517ZDEBGtest_mgs_metrics (clickhouse-client): retrieving timeseries schema from database id = e30132f4-6bad-4aa8-9298-df473cc1db82
34522024-12-20T21:46:30.517ZTRACtest_mgs_metrics (clickhouse-client): executing SQL query id = e30132f4-6bad-4aa8-9298-df473cc1db82 sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
34532024-12-20T21:46:30.591ZDEBGtest_mgs_metrics (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:38423/metrics/producers
34542024-12-20T21:46:30.591ZTRACtest_mgs_metrics (clickhouse-client): retrieved new timeseries schema id = e30132f4-6bad-4aa8-9298-df473cc1db82 n_schema = 1
34552024-12-20T21:46:30.593ZINFOtest_mgs_metrics (dropshot_internal): accepted connection local_addr = 127.0.0.1:38423 remote_addr = 127.0.0.1:54994
34562024-12-20T21:46:30.594ZTRACtest_mgs_metrics (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
34572024-12-20T21:46:30.594ZINFOtest_mgs_metrics (dropshot_external): request completed error_message_external = Timeseries not found for: hardware_component:temperature error_message_internal = Timeseries not found for: hardware_component:temperature latency_us = 206579 local_addr = 127.0.0.1:33109 method = POST remote_addr = 127.0.0.1:55320 req_id = d406625a-9dad-4322-973c-348f2b7be9e7 response_code = 400 uri = /v1/system/timeseries/query
34582024-12-20T21:46:30.595ZTRACtest_mgs_metrics (dropshot_internal): incoming request local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54994 req_id = f0dc2401-fb5a-4e64-b816-2c8b81a8857d uri = /metrics/producers
34592024-12-20T21:46:30.596ZTRACtest_mgs_metrics (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54994 req_id = f0dc2401-fb5a-4e64-b816-2c8b81a8857d resource = Database uri = /metrics/producers
34602024-12-20T21:46:30.596ZDEBGtest_mgs_metrics (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54994 req_id = f0dc2401-fb5a-4e64-b816-2c8b81a8857d roles = RoleSet { roles: {} } uri = /metrics/producers
34612024-12-20T21:46:30.597ZDEBGtest_mgs_metrics (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54994 req_id = f0dc2401-fb5a-4e64-b816-2c8b81a8857d resource = Database result = Ok(()) uri = /metrics/producers
34622024-12-20T21:46:30.597ZINFOtest_mgs_metrics (external client test context): client received response status = 400
34632024-12-20T21:46:30.932ZWARNtest_mgs_metrics (dropshot_internal): request handling cancelled (client disconnected) latency_us = 337049 local_addr = 127.0.0.1:38423 method = POST remote_addr = 127.0.0.1:54994 req_id = f0dc2401-fb5a-4e64-b816-2c8b81a8857d uri = /metrics/producers